ab_testing_online_store_v01.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project description"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Project goals\n",
    "\n",
    "***Testing changes:*** \n",
    "* The introduction of a new interface for an improved recommendation system for EU users.\n",
    "\n",
    "***Test name:***\n",
    "* interface_eu_test\n",
    "\n",
    "***Audience:***\n",
    "* The audience is new users from the EU who were encouraged to sign up in the online store. \n",
    "* Expected number of test participants: 6000\n",
    "\n",
    "***Test duration:***\n",
    "* From December 7 to December 21, 2020.\n",
    "\n",
    "***Expected result:*** \n",
    "* Increase total conversion by at least 10% at each stage of the funnel `product_page → product_cart → purchase` in test group B."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data description\n",
    "\n",
    "- `ab_project_marketing_events_us.csv` — the calendar of marketing events for 2020\n",
    "- `final_ab_new_users_us.csv` — all users who signed up in the online store from December 7 to 21, 2020\n",
    "- `final_ab_events_us.csv` — all events of the new users within the period from December 7, 2020 to January 1, 2021\n",
    "- `final_ab_participants_us.csv` — table containing test participants"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Contents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Project-description\" data-toc-modified-id=\"Project-description-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Project description</a></span><ul class=\"toc-item\"><li><span><a href=\"#Project-goals\" data-toc-modified-id=\"Project-goals-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span>Project goals</a></span></li><li><span><a href=\"#Data-description\" data-toc-modified-id=\"Data-description-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span>Data description</a></span></li><li><span><a href=\"#Table-of-Contents\" data-toc-modified-id=\"Table-of-Contents-1.3\"><span class=\"toc-item-num\">1.3&nbsp;&nbsp;</span>Table of Contents</a></span></li></ul></li><li><span><a href=\"#Project-implementation\" data-toc-modified-id=\"Project-implementation-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span>Project implementation</a></span><ul class=\"toc-item\"><li><span><a href=\"#Project-settings\" data-toc-modified-id=\"Project-settings-2.1\"><span class=\"toc-item-num\">2.1&nbsp;&nbsp;</span>Project settings</a></span><ul class=\"toc-item\"><li><span><a href=\"#Importing-libraries\" data-toc-modified-id=\"Importing-libraries-2.1.1\"><span class=\"toc-item-num\">2.1.1&nbsp;&nbsp;</span>Importing libraries</a></span></li><li><span><a href=\"#Functions---helpers-:)\" data-toc-modified-id=\"Functions---helpers-:)-2.1.2\"><span class=\"toc-item-num\">2.1.2&nbsp;&nbsp;</span>Functions - helpers :)</a></span><ul class=\"toc-item\"><li><span><a href=\"#eda_df()\" data-toc-modified-id=\"eda_df()-2.1.2.1\"><span class=\"toc-item-num\">2.1.2.1&nbsp;&nbsp;</span>eda_df()</a></span></li><li><span><a href=\"#dtype_converter()\" data-toc-modified-id=\"dtype_converter()-2.1.2.2\"><span class=\"toc-item-num\">2.1.2.2&nbsp;&nbsp;</span>dtype_converter()</a></span></li><li><span><a href=\"#time_diff()\" data-toc-modified-id=\"time_diff()-2.1.2.3\"><span class=\"toc-item-num\">2.1.2.3&nbsp;&nbsp;</span>time_diff()</a></span></li><li><span><a href=\"#get_day()\" data-toc-modified-id=\"get_day()-2.1.2.4\"><span class=\"toc-item-num\">2.1.2.4&nbsp;&nbsp;</span>get_day()</a></span></li><li><span><a href=\"#check_hypothesis()\" data-toc-modified-id=\"check_hypothesis()-2.1.2.5\"><span class=\"toc-item-num\">2.1.2.5&nbsp;&nbsp;</span>check_hypothesis()</a></span></li></ul></li><li><span><a href=\"#Data-sources\" data-toc-modified-id=\"Data-sources-2.1.3\"><span class=\"toc-item-num\">2.1.3&nbsp;&nbsp;</span>Data sources</a></span></li></ul></li><li><span><a href=\"#Data-Overview-and-Preprocessing\" data-toc-modified-id=\"Data-Overview-and-Preprocessing-2.2\"><span class=\"toc-item-num\">2.2&nbsp;&nbsp;</span>Data Overview and Preprocessing</a></span><ul class=\"toc-item\"><li><span><a href=\"#The-calendar_events-dataset\" data-toc-modified-id=\"The-calendar_events-dataset-2.2.1\"><span class=\"toc-item-num\">2.2.1&nbsp;&nbsp;</span>The <code>calendar_events</code> dataset</a></span></li><li><span><a href=\"#The-test_groups-dataset\" data-toc-modified-id=\"The-test_groups-dataset-2.2.2\"><span class=\"toc-item-num\">2.2.2&nbsp;&nbsp;</span>The <code>test_groups</code> dataset</a></span></li><li><span><a href=\"#The-new_users-dataset\" data-toc-modified-id=\"The-new_users-dataset-2.2.3\"><span class=\"toc-item-num\">2.2.3&nbsp;&nbsp;</span>The <code>new_users</code> dataset</a></span></li><li><span><a href=\"#The-events-dataset\" data-toc-modified-id=\"The-events-dataset-2.2.4\"><span class=\"toc-item-num\">2.2.4&nbsp;&nbsp;</span>The <code>events</code> dataset</a></span></li></ul></li><li><span><a href=\"#Merging-datasets\" data-toc-modified-id=\"Merging-datasets-2.3\"><span class=\"toc-item-num\">2.3&nbsp;&nbsp;</span>Merging datasets</a></span></li><li><span><a href=\"#Exploratory-Data-Analysis-(EDA)\" data-toc-modified-id=\"Exploratory-Data-Analysis-(EDA)-2.4\"><span class=\"toc-item-num\">2.4&nbsp;&nbsp;</span>Exploratory Data Analysis (EDA)</a></span><ul class=\"toc-item\"><li><span><a href=\"#Study-conversion-at-different-funnel-stages.\" data-toc-modified-id=\"Study-conversion-at-different-funnel-stages.-2.4.1\"><span class=\"toc-item-num\">2.4.1&nbsp;&nbsp;</span>Study conversion at different funnel stages.</a></span></li><li><span><a href=\"#Is-the-number-of-events-per-user-distributed-equally-in-the-samples?\" data-toc-modified-id=\"Is-the-number-of-events-per-user-distributed-equally-in-the-samples?-2.4.2\"><span class=\"toc-item-num\">2.4.2&nbsp;&nbsp;</span>Is the number of events per user distributed equally in the samples?</a></span></li><li><span><a href=\"#Are-there-users-who-enter-both-samples?\" data-toc-modified-id=\"Are-there-users-who-enter-both-samples?-2.4.3\"><span class=\"toc-item-num\">2.4.3&nbsp;&nbsp;</span>Are there users who enter both samples?</a></span></li><li><span><a href=\"#How-is-the-number-of-events-distributed-by-days?\" data-toc-modified-id=\"How-is-the-number-of-events-distributed-by-days?-2.4.4\"><span class=\"toc-item-num\">2.4.4&nbsp;&nbsp;</span>How is the number of events distributed by days?</a></span></li><li><span><a href=\"#Think-of-the-possible-details-in-the-data-that-you-have-to-take-into-account-before-starting-the-A/B-test?\" data-toc-modified-id=\"Think-of-the-possible-details-in-the-data-that-you-have-to-take-into-account-before-starting-the-A/B-test?-2.4.5\"><span class=\"toc-item-num\">2.4.5&nbsp;&nbsp;</span>Think of the possible details in the data that you have to take into account before starting the A/B test?</a></span></li><li><span><a href=\"#Conclusion:\" data-toc-modified-id=\"Conclusion:-2.4.6\"><span class=\"toc-item-num\">2.4.6&nbsp;&nbsp;</span>Conclusion:</a></span></li></ul></li><li><span><a href=\"#Evaluate-the-A/B-test-results\" data-toc-modified-id=\"Evaluate-the-A/B-test-results-2.5\"><span class=\"toc-item-num\">2.5&nbsp;&nbsp;</span>Evaluate the A/B test results</a></span><ul class=\"toc-item\"><li><span><a href=\"#What-can-you-tell-about-the-A/A-test-results?\" data-toc-modified-id=\"What-can-you-tell-about-the-A/A-test-results?-2.5.1\"><span class=\"toc-item-num\">2.5.1&nbsp;&nbsp;</span>What can you tell about the A/A test results?</a></span></li><li><span><a href=\"#Use-the-z-criterion-to-check-the-statistical-difference-between-the-proportions\" data-toc-modified-id=\"Use-the-z-criterion-to-check-the-statistical-difference-between-the-proportions-2.5.2\"><span class=\"toc-item-num\">2.5.2&nbsp;&nbsp;</span>Use the z-criterion to check the statistical difference between the proportions</a></span></li></ul></li><li><span><a href=\"#Conclusion\" data-toc-modified-id=\"Conclusion-2.6\"><span class=\"toc-item-num\">2.6&nbsp;&nbsp;</span>Conclusion</a></span></li></ul></li><li><span><a href=\"#General-conclusion\" data-toc-modified-id=\"General-conclusion-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span>General conclusion</a></span></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Project settings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os \n",
    "import sys\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import math\n",
    "from datetime import datetime\n",
    "\n",
    "from scipy import stats\n",
    "from scipy.stats import mode\n",
    "\n",
    "# visualization\n",
    "import plotly.express as px\n",
    "import plotly.io as pio\n",
    "pio.templates.default = \"none\"\n",
    "\n",
    "# to show warnings only once:\n",
    "import warnings; warnings.filterwarnings(action='once')\n",
    "\n",
    "# avoid warnings (if necessary)\n",
    "import warnings\n",
    "if not sys.warnoptions:\n",
    "       warnings.simplefilter(\"ignore\")\n",
    "\n",
    "# to print all the outputs in the cell\n",
    "from IPython.core.interactiveshell import InteractiveShell\n",
    "InteractiveShell.ast_node_interactivity = \"all\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functions - helpers :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### eda_df()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def eda_df(df):\n",
    "    \"\"\"\n",
    "    The function takes a data frame as input and calculates summary statistics\n",
    "    to reveal the understanding of the data in each column.\n",
    "    The function returns a data frame where each feature in the data set is a row and the summary statistics are columns. \n",
    "    \"\"\"\n",
    "    \n",
    "    print('----------------- Summary statistics -----------------')\n",
    "    eda = {}\n",
    "    eda['dtype'] = df.dtypes\n",
    "#     eda['mamory_usage'] = df.memory_usage('deep')\n",
    "    eda['count'] = df.count()\n",
    "    eda['min'] = df.min()\n",
    "    eda['mean'] = df.mean()\n",
    "    eda['median'] = df.median()    \n",
    "    eda['max'] = df.max()\n",
    "    eda['std'] = df.std()\n",
    "    eda['skew'] = df.skew(skipna=True)\n",
    "    eda['null_sum'] = df.isnull().sum()\n",
    "    eda['null_%'] = round((df.isnull().mean() * 100), 2)\n",
    "\n",
    "    return pd.DataFrame(eda)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### dtype_converter()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def dtype_converter(series, dtype):\n",
    "    \"\"\"\n",
    "    The function converts data type of series to another.\n",
    "    And returns series.\n",
    "    \n",
    "    \"\"\"\n",
    "    \n",
    "    types_data = ['int', 'float', 'float64', 'bool', 'str', 'category']\n",
    "    try:\n",
    "        if any(elem in dtype for elem in types_data):\n",
    "            series = series.astype(dtype)      \n",
    "        elif dtype == 'date':\n",
    "            series = pd.to_datetime(series, format='%Y-%m-%dT%H:%M:%S')\n",
    "        elif dtype == 'numeric':\n",
    "            pd.to_numeric(series, errors='ignore')\n",
    "    except ValueError:\n",
    "        pass\n",
    "\n",
    "    return series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### time_diff()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def time_diff(end_ts, start_ts, time_val):\n",
    "    \"\"\"\n",
    "    The function returns a difference between two timestamp in time_val\n",
    "    \"\"\"\n",
    "    \n",
    "    try:\n",
    "        if time_val == 'seconds':\n",
    "            return (end_ts - start_ts).dt.seconds\n",
    "        elif time_val == 'days':\n",
    "            return (end_ts - start_ts).dt.days \n",
    "        elif time_val == 'week':\n",
    "            return (end_ts - start_ts).dt.week \n",
    "        elif time_val == 'month':\n",
    "            return (end_ts - start_ts).dt.month\n",
    "        elif time_val == 'year':\n",
    "            return (end_ts - start_ts).dt.year\n",
    "        \n",
    "    except ValueError:\n",
    "        'Data type is not datetime64[ns]'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  get_day()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_day(df, date_column): \n",
    "    \"\"\"\n",
    "    The function retrives day from the `date_column` and returns it in datetime64[ns] format.\n",
    "    \"\"\"\n",
    "    \n",
    "    return pd.to_datetime(df[date_column].dt.date)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### check_hypothesis()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def check_hypothesis(sample1, sample2, event, alpha):\n",
    "\n",
    "    # let's start with successes, using\n",
    "\n",
    "    successes1 = users_pivot[users_pivot.event_name == event][sample1].iloc[0]\n",
    "    successes2 = users_pivot[users_pivot.event_name == event][sample2].iloc[0]\n",
    "\n",
    "    # for trials we can go back to original df or used a pre-aggregated data\n",
    "    trials1 = df_ab_testing[df_ab_testing.group == sample1]['user_id'].nunique()\n",
    "    trials2 = df_ab_testing[df_ab_testing.group == sample2]['user_id'].nunique()\n",
    "\n",
    "    # proportion for success in the first group\n",
    "    p1 = successes1 / trials1\n",
    "\n",
    "    # proportion for success in the second group\n",
    "    p2 = successes2 / trials2\n",
    "\n",
    "    # proportion in a combined dataset\n",
    "    p_combined = (successes1 + successes2) / (trials1 + trials2)\n",
    "\n",
    "    difference = p1 - p2\n",
    "\n",
    "    # calculating the statistic in standard deviations of the standard normal distribution\n",
    "    z_value = difference / math.sqrt(p_combined * (1 - p_combined) * (1 / trials1 + 1 / trials2))\n",
    "\n",
    "    # setting up the standard normal distribution (mean 0, standard deviation 1)\n",
    "    distr = stats.norm(0, 1)\n",
    "\n",
    "    p_value = (1 - distr.cdf(abs(z_value))) * 2\n",
    "\n",
    "    print('p-value: ', p_value)\n",
    "\n",
    "    if (p_value < alpha):\n",
    "        print(\"Reject H0 for\", event, 'and groups', sample1, sample2)\n",
    "    else:\n",
    "        print(\"Fail to Reject H0 for\", event, 'and groups', sample1, sample2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Data sources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# source_dir = '/datasets/'\n",
    "source_dir = 'datasets/' # works for me:)\n",
    "\n",
    "missing_values = [\"n/a\", \"na\", \"--\"]\n",
    "\n",
    "# filenames\n",
    "calendar_events = 'ab_project_marketing_events_us.csv'\n",
    "test_groups = 'final_ab_participants_us.csv'\n",
    "new_users = 'final_ab_new_users_us.csv'\n",
    "events = 'final_ab_events_us.csv'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Overview and Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The `calendar_events` dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Downloading:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_calendar_events = pd.read_csv(os.path.join(source_dir, calendar_events),\n",
    "                        parse_dates=['start_dt', 'finish_dt'],\n",
    "                        na_values=missing_values)\n",
    "\n",
    "# saving the original data set for comparing changes to be made\n",
    "orig_df_calendar_events = df_calendar_events.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Overview and descriptive statistic:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- The number of rows and columns -----------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(14, 4)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- A concise summary of the DataFrame -----------------\n",
      "\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 14 entries, 0 to 13\n",
      "Data columns (total 4 columns):\n",
      " #   Column     Non-Null Count  Dtype         \n",
      "---  ------     --------------  -----         \n",
      " 0   name       14 non-null     object        \n",
      " 1   regions    14 non-null     object        \n",
      " 2   start_dt   14 non-null     datetime64[ns]\n",
      " 3   finish_dt  14 non-null     datetime64[ns]\n",
      "dtypes: datetime64[ns](2), object(2)\n",
      "memory usage: 576.0+ bytes\n",
      "\n",
      "----------------- Dataset -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>regions</th>\n",
       "      <th>start_dt</th>\n",
       "      <th>finish_dt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Chinese New Year Promo</td>\n",
       "      <td>APAC</td>\n",
       "      <td>2020-01-25</td>\n",
       "      <td>2020-02-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>St. Valentine's Day Giveaway</td>\n",
       "      <td>EU, CIS, APAC, N.America</td>\n",
       "      <td>2020-02-14</td>\n",
       "      <td>2020-02-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>International Women's Day Promo</td>\n",
       "      <td>EU, CIS, APAC</td>\n",
       "      <td>2020-03-08</td>\n",
       "      <td>2020-03-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>St. Patric's Day Promo</td>\n",
       "      <td>EU, N.America</td>\n",
       "      <td>2020-03-17</td>\n",
       "      <td>2020-03-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Easter Promo</td>\n",
       "      <td>EU, CIS, APAC, N.America</td>\n",
       "      <td>2020-04-12</td>\n",
       "      <td>2020-04-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Labor day (May 1st) Ads Campaign</td>\n",
       "      <td>EU, CIS, APAC</td>\n",
       "      <td>2020-05-01</td>\n",
       "      <td>2020-05-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Victory Day CIS (May 9th) Event</td>\n",
       "      <td>CIS</td>\n",
       "      <td>2020-05-09</td>\n",
       "      <td>2020-05-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Dragon Boat Festival Giveaway</td>\n",
       "      <td>APAC</td>\n",
       "      <td>2020-06-25</td>\n",
       "      <td>2020-07-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4th of July Promo</td>\n",
       "      <td>N.America</td>\n",
       "      <td>2020-07-04</td>\n",
       "      <td>2020-07-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Chinese Moon Festival</td>\n",
       "      <td>APAC</td>\n",
       "      <td>2020-10-01</td>\n",
       "      <td>2020-10-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Single's Day Gift Promo</td>\n",
       "      <td>APAC</td>\n",
       "      <td>2020-11-11</td>\n",
       "      <td>2020-11-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Black Friday Ads Campaign</td>\n",
       "      <td>EU, CIS, APAC, N.America</td>\n",
       "      <td>2020-11-26</td>\n",
       "      <td>2020-12-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Christmas&amp;New Year Promo</td>\n",
       "      <td>EU, N.America</td>\n",
       "      <td>2020-12-25</td>\n",
       "      <td>2021-01-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>CIS New Year Gift Lottery</td>\n",
       "      <td>CIS</td>\n",
       "      <td>2020-12-30</td>\n",
       "      <td>2021-01-07</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                name                   regions   start_dt  \\\n",
       "6             Chinese New Year Promo                      APAC 2020-01-25   \n",
       "1       St. Valentine's Day Giveaway  EU, CIS, APAC, N.America 2020-02-14   \n",
       "8    International Women's Day Promo             EU, CIS, APAC 2020-03-08   \n",
       "2             St. Patric's Day Promo             EU, N.America 2020-03-17   \n",
       "3                       Easter Promo  EU, CIS, APAC, N.America 2020-04-12   \n",
       "7   Labor day (May 1st) Ads Campaign             EU, CIS, APAC 2020-05-01   \n",
       "9    Victory Day CIS (May 9th) Event                       CIS 2020-05-09   \n",
       "11     Dragon Boat Festival Giveaway                      APAC 2020-06-25   \n",
       "4                  4th of July Promo                 N.America 2020-07-04   \n",
       "13             Chinese Moon Festival                      APAC 2020-10-01   \n",
       "12           Single's Day Gift Promo                      APAC 2020-11-11   \n",
       "5          Black Friday Ads Campaign  EU, CIS, APAC, N.America 2020-11-26   \n",
       "0           Christmas&New Year Promo             EU, N.America 2020-12-25   \n",
       "10         CIS New Year Gift Lottery                       CIS 2020-12-30   \n",
       "\n",
       "    finish_dt  \n",
       "6  2020-02-07  \n",
       "1  2020-02-16  \n",
       "8  2020-03-10  \n",
       "2  2020-03-19  \n",
       "3  2020-04-19  \n",
       "7  2020-05-03  \n",
       "9  2020-05-11  \n",
       "11 2020-07-01  \n",
       "4  2020-07-11  \n",
       "13 2020-10-07  \n",
       "12 2020-11-12  \n",
       "5  2020-12-01  \n",
       "0  2021-01-03  \n",
       "10 2021-01-07  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('\\n----------------- The number of rows and columns -----------------')\n",
    "df_calendar_events.shape\n",
    "print('\\n----------------- A concise summary of the DataFrame -----------------\\n')\n",
    "df_calendar_events.info()\n",
    "\n",
    "print('\\n----------------- Dataset -----------------')\n",
    "df_calendar_events = df_calendar_events.sort_values(by='start_dt')\n",
    "df_calendar_events"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>regions</th>\n",
       "      <th>start_dt</th>\n",
       "      <th>finish_dt</th>\n",
       "      <th>n_days</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Chinese New Year Promo</td>\n",
       "      <td>APAC</td>\n",
       "      <td>2020-01-25</td>\n",
       "      <td>2020-02-07</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>St. Valentine's Day Giveaway</td>\n",
       "      <td>EU, CIS, APAC, N.America</td>\n",
       "      <td>2020-02-14</td>\n",
       "      <td>2020-02-16</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>International Women's Day Promo</td>\n",
       "      <td>EU, CIS, APAC</td>\n",
       "      <td>2020-03-08</td>\n",
       "      <td>2020-03-10</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>St. Patric's Day Promo</td>\n",
       "      <td>EU, N.America</td>\n",
       "      <td>2020-03-17</td>\n",
       "      <td>2020-03-19</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Easter Promo</td>\n",
       "      <td>EU, CIS, APAC, N.America</td>\n",
       "      <td>2020-04-12</td>\n",
       "      <td>2020-04-19</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Labor day (May 1st) Ads Campaign</td>\n",
       "      <td>EU, CIS, APAC</td>\n",
       "      <td>2020-05-01</td>\n",
       "      <td>2020-05-03</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Victory Day CIS (May 9th) Event</td>\n",
       "      <td>CIS</td>\n",
       "      <td>2020-05-09</td>\n",
       "      <td>2020-05-11</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Dragon Boat Festival Giveaway</td>\n",
       "      <td>APAC</td>\n",
       "      <td>2020-06-25</td>\n",
       "      <td>2020-07-01</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4th of July Promo</td>\n",
       "      <td>N.America</td>\n",
       "      <td>2020-07-04</td>\n",
       "      <td>2020-07-11</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Chinese Moon Festival</td>\n",
       "      <td>APAC</td>\n",
       "      <td>2020-10-01</td>\n",
       "      <td>2020-10-07</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Single's Day Gift Promo</td>\n",
       "      <td>APAC</td>\n",
       "      <td>2020-11-11</td>\n",
       "      <td>2020-11-12</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Black Friday Ads Campaign</td>\n",
       "      <td>EU, CIS, APAC, N.America</td>\n",
       "      <td>2020-11-26</td>\n",
       "      <td>2020-12-01</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Christmas&amp;New Year Promo</td>\n",
       "      <td>EU, N.America</td>\n",
       "      <td>2020-12-25</td>\n",
       "      <td>2021-01-03</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>CIS New Year Gift Lottery</td>\n",
       "      <td>CIS</td>\n",
       "      <td>2020-12-30</td>\n",
       "      <td>2021-01-07</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                name                   regions   start_dt  \\\n",
       "6             Chinese New Year Promo                      APAC 2020-01-25   \n",
       "1       St. Valentine's Day Giveaway  EU, CIS, APAC, N.America 2020-02-14   \n",
       "8    International Women's Day Promo             EU, CIS, APAC 2020-03-08   \n",
       "2             St. Patric's Day Promo             EU, N.America 2020-03-17   \n",
       "3                       Easter Promo  EU, CIS, APAC, N.America 2020-04-12   \n",
       "7   Labor day (May 1st) Ads Campaign             EU, CIS, APAC 2020-05-01   \n",
       "9    Victory Day CIS (May 9th) Event                       CIS 2020-05-09   \n",
       "11     Dragon Boat Festival Giveaway                      APAC 2020-06-25   \n",
       "4                  4th of July Promo                 N.America 2020-07-04   \n",
       "13             Chinese Moon Festival                      APAC 2020-10-01   \n",
       "12           Single's Day Gift Promo                      APAC 2020-11-11   \n",
       "5          Black Friday Ads Campaign  EU, CIS, APAC, N.America 2020-11-26   \n",
       "0           Christmas&New Year Promo             EU, N.America 2020-12-25   \n",
       "10         CIS New Year Gift Lottery                       CIS 2020-12-30   \n",
       "\n",
       "    finish_dt  n_days  \n",
       "6  2020-02-07      13  \n",
       "1  2020-02-16       2  \n",
       "8  2020-03-10       2  \n",
       "2  2020-03-19       2  \n",
       "3  2020-04-19       7  \n",
       "7  2020-05-03       2  \n",
       "9  2020-05-11       2  \n",
       "11 2020-07-01       6  \n",
       "4  2020-07-11       7  \n",
       "13 2020-10-07       6  \n",
       "12 2020-11-12       1  \n",
       "5  2020-12-01       5  \n",
       "0  2021-01-03       9  \n",
       "10 2021-01-07       8  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_calendar_events['n_days'] = time_diff(df_calendar_events.finish_dt, df_calendar_events.start_dt, 'days')\n",
    "df_calendar_events"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Preliminary overview:\n",
    "* The `df_calendar_events` dataset contains 14 events over the year 2020.\n",
    "* The longest event duration is 13 days and belongs to 'Chinese New Year Promo'.\n",
    "* The shortest event is 1 day. This is 'Single's Day Gift Promo'.\n",
    "* There are no marketing events in August and September.\n",
    "* The 'CIS New Year Gift Lottery' event coincides with the 'Christmas&New Year Promo' event. They overlap from December 30 to January 3 inclusive.\n",
    "* There is no null values.\n",
    "* **The 14 days of A / B testing from December 7 to 21 did not coincide with any marketing event.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The `test_groups` dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Downloading:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_test_groups = pd.read_csv(os.path.join(source_dir, test_groups),\n",
    "                             na_values=missing_values,\n",
    "                             dtype={\n",
    "                                 'ab_test': 'category',\n",
    "                                 'group': 'category'\n",
    "                             })\n",
    "\n",
    "# saving the original data set for comparing changes to be made\n",
    "orig_df_test_groups = df_test_groups.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Overview and descriptive statistic:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------- Summary statistics -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dtype</th>\n",
       "      <th>count</th>\n",
       "      <th>min</th>\n",
       "      <th>mean</th>\n",
       "      <th>median</th>\n",
       "      <th>max</th>\n",
       "      <th>std</th>\n",
       "      <th>skew</th>\n",
       "      <th>null_sum</th>\n",
       "      <th>null_%</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ab_test</th>\n",
       "      <td>category</td>\n",
       "      <td>18268</td>\n",
       "      <td>interface_eu_test</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>recommender_system_test</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>group</th>\n",
       "      <td>category</td>\n",
       "      <td>18268</td>\n",
       "      <td>A</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>B</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user_id</th>\n",
       "      <td>object</td>\n",
       "      <td>18268</td>\n",
       "      <td>0002CE61FF2C4011</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FFF58BC33966EB51</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            dtype  count                min  mean  median  \\\n",
       "ab_test  category  18268  interface_eu_test   NaN     NaN   \n",
       "group    category  18268                  A   NaN     NaN   \n",
       "user_id    object  18268   0002CE61FF2C4011   NaN     NaN   \n",
       "\n",
       "                             max  std  skew  null_sum  null_%  \n",
       "ab_test  recommender_system_test  NaN   NaN         0     0.0  \n",
       "group                          B  NaN   NaN         0     0.0  \n",
       "user_id         FFF58BC33966EB51  NaN   NaN         0     0.0  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset head -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>group</th>\n",
       "      <th>ab_test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>D1ABA3E2887B6A73</td>\n",
       "      <td>A</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A7A3664BD6242119</td>\n",
       "      <td>A</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>DABC14FDDFADD29E</td>\n",
       "      <td>A</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>04988C5DF189632E</td>\n",
       "      <td>A</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>482F14783456D21B</td>\n",
       "      <td>B</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            user_id group                  ab_test\n",
       "0  D1ABA3E2887B6A73     A  recommender_system_test\n",
       "1  A7A3664BD6242119     A  recommender_system_test\n",
       "2  DABC14FDDFADD29E     A  recommender_system_test\n",
       "3  04988C5DF189632E     A  recommender_system_test\n",
       "4  482F14783456D21B     B  recommender_system_test"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset tail -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>group</th>\n",
       "      <th>ab_test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>18263</th>\n",
       "      <td>1D302F8688B91781</td>\n",
       "      <td>B</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18264</th>\n",
       "      <td>3DE51B726983B657</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18265</th>\n",
       "      <td>F501F79D332BE86C</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18266</th>\n",
       "      <td>63FBE257B05F2245</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18267</th>\n",
       "      <td>79F9ABFB029CF724</td>\n",
       "      <td>B</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                user_id group            ab_test\n",
       "18263  1D302F8688B91781     B  interface_eu_test\n",
       "18264  3DE51B726983B657     A  interface_eu_test\n",
       "18265  F501F79D332BE86C     A  interface_eu_test\n",
       "18266  63FBE257B05F2245     A  interface_eu_test\n",
       "18267  79F9ABFB029CF724     B  interface_eu_test"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset sample -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>group</th>\n",
       "      <th>ab_test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4340</th>\n",
       "      <td>D755F72C453BF6A7</td>\n",
       "      <td>B</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9979</th>\n",
       "      <td>B2240C185E9225C9</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8875</th>\n",
       "      <td>EBA744CC064A73C6</td>\n",
       "      <td>B</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2296</th>\n",
       "      <td>E4EB5A73A37A55CA</td>\n",
       "      <td>B</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>166</th>\n",
       "      <td>F63D747F02F31C09</td>\n",
       "      <td>B</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               user_id group                  ab_test\n",
       "4340  D755F72C453BF6A7     B  recommender_system_test\n",
       "9979  B2240C185E9225C9     A        interface_eu_test\n",
       "8875  EBA744CC064A73C6     B        interface_eu_test\n",
       "2296  E4EB5A73A37A55CA     B  recommender_system_test\n",
       "166   F63D747F02F31C09     B  recommender_system_test"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- The number of rows, columns -----------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(18268, 3)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eda_df(df_test_groups)\n",
    "print('\\n----------------- Dataset head -----------------')\n",
    "df_test_groups.head()\n",
    "print('\\n----------------- Dataset tail -----------------')\n",
    "df_test_groups.tail()\n",
    "print('\\n----------------- Dataset sample -----------------')\n",
    "df_test_groups.sample(5)\n",
    "print('\\n----------------- The number of rows, columns -----------------')\n",
    "df_test_groups.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Count unique values in the columns:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "user_id    16666\n",
       "group          2\n",
       "ab_test        2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>group</th>\n",
       "      <th>ab_test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  group                  ab_test\n",
       "0     A  recommender_system_test\n",
       "1     B        interface_eu_test"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test_groups.apply(lambda x: len(x.unique()))\n",
    "\n",
    "df_test_groups[['group', 'ab_test']].apply(lambda x: x.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test_groups.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Preliminary overview:\n",
    "* The `test_groups` dataset contains 18,268 records\n",
    "* All users are divided into 2 test groups and 2 types of A / B tests.\n",
    "* There are no duplicates and Nan values.\n",
    "* However, there are noticeably fewer unique test participants (16,666) than entries. This means that we have users in the dataset who passed both tests, `recommender_system_test` and in `interface_eu_test`. \n",
    "\n",
    "*I think we need to delete users who participate in two tests because we don't know which version of the interface the participants in the second test saw.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>ab_test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>001064FEAAB631A1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>00341D8401F0F665</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>003B6786B4FF5B03</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0082295A41A867B5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>00E68F103C66C1F7</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             user_id  ab_test\n",
       "2   001064FEAAB631A1        2\n",
       "10  00341D8401F0F665        2\n",
       "12  003B6786B4FF5B03        2\n",
       "29  0082295A41A867B5        2\n",
       "46  00E68F103C66C1F7        2"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "1602"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>group</th>\n",
       "      <th>ab_test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4768</th>\n",
       "      <td>0082295A41A867B5</td>\n",
       "      <td>A</td>\n",
       "      <td>recommender_system_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14161</th>\n",
       "      <td>0082295A41A867B5</td>\n",
       "      <td>B</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                user_id group                  ab_test\n",
       "4768   0082295A41A867B5     A  recommender_system_test\n",
       "14161  0082295A41A867B5     B        interface_eu_test"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participants_of_2_tests = df_test_groups.groupby(['user_id'])['ab_test'].nunique().reset_index().query('ab_test == 2')\n",
    "participants_of_2_tests.head()\n",
    "participants_of_2_tests.shape[0]\n",
    "\n",
    "df_test_groups.query('user_id == \"0082295A41A867B5\"')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*As we can see, the dataset contains 1602 records of users who participate in both tests. Let's remove them.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "18268"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>group</th>\n",
       "      <th>ab_test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [user_id, group, ab_test]\n",
       "Index: []"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "15064"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test_groups.shape[0]\n",
    "\n",
    "# Create a list of user ids to remove\n",
    "lst_users_to_remove = participants_of_2_tests.user_id.to_list()\n",
    "\n",
    "# Removing users who are in both A and B test groups\n",
    "df_test_groups = df_test_groups[~df_test_groups.user_id.isin(lst_users_to_remove)]\n",
    "\n",
    "# Verifying the result:\n",
    "df_test_groups.query('user_id == \"lst_users_to_remove[0]\"')\n",
    "df_test_groups.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*The `test_groups` dataset has 2 types of A/B test:*\n",
    "- recommender_system_test,\n",
    "- interface_eu_test.\n",
    "\n",
    "*For the purposes of our current project, we have to use data related to `interface_eu_test` test.* \n",
    "\n",
    "*So let's split the dataset to get the necessary data and study it.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------- Summary statistics -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dtype</th>\n",
       "      <th>count</th>\n",
       "      <th>min</th>\n",
       "      <th>mean</th>\n",
       "      <th>median</th>\n",
       "      <th>max</th>\n",
       "      <th>std</th>\n",
       "      <th>skew</th>\n",
       "      <th>null_sum</th>\n",
       "      <th>null_%</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ab_test</th>\n",
       "      <td>category</td>\n",
       "      <td>9965</td>\n",
       "      <td>interface_eu_test</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>interface_eu_test</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>group</th>\n",
       "      <td>category</td>\n",
       "      <td>9965</td>\n",
       "      <td>A</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>B</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user_id</th>\n",
       "      <td>object</td>\n",
       "      <td>9965</td>\n",
       "      <td>0002CE61FF2C4011</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FFF58BC33966EB51</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            dtype  count                min  mean  median                max  \\\n",
       "ab_test  category   9965  interface_eu_test   NaN     NaN  interface_eu_test   \n",
       "group    category   9965                  A   NaN     NaN                  B   \n",
       "user_id    object   9965   0002CE61FF2C4011   NaN     NaN   FFF58BC33966EB51   \n",
       "\n",
       "         std  skew  null_sum  null_%  \n",
       "ab_test  NaN   NaN         0     0.0  \n",
       "group    NaN   NaN         0     0.0  \n",
       "user_id  NaN   NaN         0     0.0  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset head -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>group</th>\n",
       "      <th>ab_test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6701</th>\n",
       "      <td>D4E530F6595A05A3</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6703</th>\n",
       "      <td>6BCB0F33D3BAB8C2</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6707</th>\n",
       "      <td>2D2E7AA539CF348F</td>\n",
       "      <td>B</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6708</th>\n",
       "      <td>4BA448BCE1343C6F</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6709</th>\n",
       "      <td>76B6CDF2A8B1DBFC</td>\n",
       "      <td>B</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               user_id group            ab_test\n",
       "6701  D4E530F6595A05A3     A  interface_eu_test\n",
       "6703  6BCB0F33D3BAB8C2     A  interface_eu_test\n",
       "6707  2D2E7AA539CF348F     B  interface_eu_test\n",
       "6708  4BA448BCE1343C6F     A  interface_eu_test\n",
       "6709  76B6CDF2A8B1DBFC     B  interface_eu_test"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- The number of rows, columns -----------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(9965, 3)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "group\n",
       "A    5012\n",
       "B    4953\n",
       "Name: user_id, dtype: int64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_eu_test = df_test_groups.query('ab_test == \"interface_eu_test\"')\n",
    "eda_df(df_eu_test)\n",
    "\n",
    "print('\\n----------------- Dataset head -----------------')\n",
    "df_eu_test.head()\n",
    "print('\\n----------------- The number of rows, columns -----------------')\n",
    "df_eu_test.shape\n",
    "\n",
    "df_eu_test.groupby('group')['user_id'].nunique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Now let's check if the dataset has users who are in both test groups A and B.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>group</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [user_id, group]\n",
       "Index: []"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participants_of_2_group = df_eu_test.groupby(['user_id'])['group'].nunique().reset_index().query('group == 2')\n",
    "participants_of_2_group.head()\n",
    "participants_of_2_group.shape[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>group</th>\n",
       "      <th>n_users</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>5012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B</td>\n",
       "      <td>4953</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  group  n_users\n",
       "0     A     5012\n",
       "1     B     4953"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"450\" style=\"\" viewBox=\"0 0 700 450\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"450\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-ef51c5\"><g class=\"clips\"/><g class=\"gradients\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"/><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"><g class=\"trace\" stroke-linejoin=\"round\" style=\"opacity: 1;\"><g class=\"slice\"><path class=\"surface\" d=\"M350,235l0,-135a135,135 0 1 1 -2.5109224906842167,269.9766471218106Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(417.49708082710356,240.4246327713151)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\">50.3%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M350,235l-2.5109224906841896,134.97664712181057a135,135 0 0 1 2.5109224906841896,-269.9766471218106Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(282.50145961801024,239.16910365406937)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">49.7%</text></g></g></g></g><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-ef51c5\"><g class=\"clips\"/><clipPath id=\"legendef51c5\"><rect width=\"55\" height=\"48\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(630.8, 100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\" width=\"55\" height=\"48\" x=\"0\" y=\"0\"/><g class=\"scrollbox\" transform=\"translate(0, 0)\" clip-path=\"url('#legendef51c5')\"><g class=\"groups\"><g class=\"traces\" style=\"opacity: 1;\" transform=\"translate(0, 14.5)\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">A</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"49.625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" style=\"opacity: 1;\" transform=\"translate(0, 33.5)\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">B</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"49.625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\" x=\"0\" y=\"0\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"350\" y=\"50\" text-anchor=\"middle\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">The ratio of users of 2 test groups.</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tmp_by_group = df_eu_test.groupby('group')['user_id'].agg([\n",
    "    ('n_users', 'count')\n",
    "]).sort_values(by='n_users', ascending=False).reset_index()\n",
    "\n",
    "tmp_by_group\n",
    "\n",
    "fig = px.pie(\n",
    "    tmp_by_group,\n",
    "    values='n_users',\n",
    "    names='group',\n",
    "    #              color='region',\n",
    "    title='The ratio of users of 2 test groups.')\n",
    "\n",
    "fig.show('svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Conclusion:\n",
    "* After preprocessing, we have a dataset named `df_eu_test`, which contains the users and groups of data we need,` interface_eu_test` A / B test.\n",
    "* The dataset `df_eu_test` has 9965 unique participants which are almost equally split:\n",
    "      * 5012 users - in group A,\n",
    "      * 4953 users - in group B\n",
    "* In our dataset we do not have users that got into both test types or into both test groups."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The `new_users` dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Downloading:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_new_users = pd.read_csv(os.path.join(source_dir, new_users),\n",
    "                           parse_dates=['first_date'],\n",
    "                           na_values=missing_values,\n",
    "                           dtype={\n",
    "                               'region': 'category',\n",
    "                               'device': 'category'\n",
    "                           })\n",
    "\n",
    "# saving the original data set for comparing changes to be made\n",
    "orig_df_new_users = df_new_users.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Overview and descriptive statistic:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------- Summary statistics -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dtype</th>\n",
       "      <th>count</th>\n",
       "      <th>min</th>\n",
       "      <th>mean</th>\n",
       "      <th>median</th>\n",
       "      <th>max</th>\n",
       "      <th>std</th>\n",
       "      <th>skew</th>\n",
       "      <th>null_sum</th>\n",
       "      <th>null_%</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>device</th>\n",
       "      <td>category</td>\n",
       "      <td>61733</td>\n",
       "      <td>Android</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>iPhone</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>first_date</th>\n",
       "      <td>datetime64[ns]</td>\n",
       "      <td>61733</td>\n",
       "      <td>2020-12-07 00:00:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2020-12-23 00:00:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <td>category</td>\n",
       "      <td>61733</td>\n",
       "      <td>APAC</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N.America</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user_id</th>\n",
       "      <td>object</td>\n",
       "      <td>61733</td>\n",
       "      <td>0001710F4DDB1D1B</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FFFFE36C0F6E92DF</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     dtype  count                  min  mean  median  \\\n",
       "device            category  61733              Android   NaN     NaN   \n",
       "first_date  datetime64[ns]  61733  2020-12-07 00:00:00   NaN     NaN   \n",
       "region            category  61733                 APAC   NaN     NaN   \n",
       "user_id             object  61733     0001710F4DDB1D1B   NaN     NaN   \n",
       "\n",
       "                            max  std  skew  null_sum  null_%  \n",
       "device                   iPhone  NaN   NaN         0     0.0  \n",
       "first_date  2020-12-23 00:00:00  NaN   NaN         0     0.0  \n",
       "region                N.America  NaN   NaN         0     0.0  \n",
       "user_id        FFFFE36C0F6E92DF  NaN   NaN         0     0.0  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset head -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>first_date</th>\n",
       "      <th>region</th>\n",
       "      <th>device</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>D72A72121175D8BE</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>F1C668619DFE6E65</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>N.America</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2E1BF1D4C37EA01F</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>50734A22C0C63768</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>E1BDDCE0DAFA2679</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>N.America</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            user_id first_date     region   device\n",
       "0  D72A72121175D8BE 2020-12-07         EU       PC\n",
       "1  F1C668619DFE6E65 2020-12-07  N.America  Android\n",
       "2  2E1BF1D4C37EA01F 2020-12-07         EU       PC\n",
       "3  50734A22C0C63768 2020-12-07         EU   iPhone\n",
       "4  E1BDDCE0DAFA2679 2020-12-07  N.America   iPhone"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset tail -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>first_date</th>\n",
       "      <th>region</th>\n",
       "      <th>device</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>61728</th>\n",
       "      <td>1DB53B933257165D</td>\n",
       "      <td>2020-12-20</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61729</th>\n",
       "      <td>538643EB4527ED03</td>\n",
       "      <td>2020-12-20</td>\n",
       "      <td>EU</td>\n",
       "      <td>Mac</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61730</th>\n",
       "      <td>7ADEE837D5D8CBBD</td>\n",
       "      <td>2020-12-20</td>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61731</th>\n",
       "      <td>1C7D23927835213F</td>\n",
       "      <td>2020-12-20</td>\n",
       "      <td>EU</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61732</th>\n",
       "      <td>8F04273BB2860229</td>\n",
       "      <td>2020-12-20</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                user_id first_date region   device\n",
       "61728  1DB53B933257165D 2020-12-20     EU  Android\n",
       "61729  538643EB4527ED03 2020-12-20     EU      Mac\n",
       "61730  7ADEE837D5D8CBBD 2020-12-20     EU       PC\n",
       "61731  1C7D23927835213F 2020-12-20     EU   iPhone\n",
       "61732  8F04273BB2860229 2020-12-20     EU  Android"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset sample -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>first_date</th>\n",
       "      <th>region</th>\n",
       "      <th>device</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>39303</th>\n",
       "      <td>D072F407D6CE2C45</td>\n",
       "      <td>2020-12-17</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29747</th>\n",
       "      <td>212C2AE7F88FD838</td>\n",
       "      <td>2020-12-16</td>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11319</th>\n",
       "      <td>8163B5D7F260AF0A</td>\n",
       "      <td>2020-12-21</td>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18452</th>\n",
       "      <td>CB3A7EBF283D2E7B</td>\n",
       "      <td>2020-12-08</td>\n",
       "      <td>N.America</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51978</th>\n",
       "      <td>1A32EEE6F8C87C3B</td>\n",
       "      <td>2020-12-19</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                user_id first_date     region   device\n",
       "39303  D072F407D6CE2C45 2020-12-17         EU  Android\n",
       "29747  212C2AE7F88FD838 2020-12-16         EU       PC\n",
       "11319  8163B5D7F260AF0A 2020-12-21         EU       PC\n",
       "18452  CB3A7EBF283D2E7B 2020-12-08  N.America       PC\n",
       "51978  1A32EEE6F8C87C3B 2020-12-19         EU  Android"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- The number of rows, columns -----------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(61733, 4)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eda_df(df_new_users)\n",
    "print('\\n----------------- Dataset head -----------------')\n",
    "df_new_users.head()\n",
    "print('\\n----------------- Dataset tail -----------------')\n",
    "df_new_users.tail()\n",
    "print('\\n----------------- Dataset sample -----------------')\n",
    "df_new_users.sample(5)\n",
    "print('\\n----------------- The number of rows, columns -----------------')\n",
    "df_new_users.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Count unique values in the columns:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "user_id       61733\n",
       "first_date       17\n",
       "region            4\n",
       "device            4\n",
       "dtype: int64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new_users.apply(lambda x: len(x.unique()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>region</th>\n",
       "      <th>device</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>N.America</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>APAC</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CIS</td>\n",
       "      <td>Mac</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      region   device\n",
       "0         EU       PC\n",
       "1  N.America  Android\n",
       "2       APAC   iPhone\n",
       "3        CIS      Mac"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new_users[['region', 'device']].apply(lambda x: x.unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Checking duplicates:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new_users.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Preliminary overview:\n",
    "* The `new_users` dataset contains 61733 unique entries.\n",
    "* New users signed up in the online store from December 7 to 23, 2020.\n",
    "* New users have come from 4 regions: EU, N.America, APAC, CIS.\n",
    "* They utilize 4 types of devices: PC, Android, iPhone, Mac. \n",
    "* There are no duplicates and Nan values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*To test the new interface of the online store, we have 2 mandatory testing conditions:*\n",
    "* Testing lasted 14 days, from December 7 to December 21. So these days are included in the scope of the project;\n",
    "* We have to research new EU users.\n",
    "\n",
    "*Firstly, let's look at the days of December 22nd and 23rd as outside the scope of our task and remove them.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-12-21 00:00:00')"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new_users = df_new_users.query('first_date < \"2020-12-22\"')\n",
    "\n",
    "# Verifying the result\n",
    "df_new_users.first_date.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Secondly, we have to extract data on new EU users from the dataset. Moreover, they must be our test participants.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List of new EU users who participate in our test\n",
    "lst_eu_test_users = df_eu_test.user_id.to_list()\n",
    "\n",
    "# Creating df of users who participate in our test\n",
    "df_new_eu_users = df_new_users[(df_new_users['user_id'].isin(lst_eu_test_users)) & (df_new_users['region'] == \"EU\")]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Let's explore the `df_new_eu_users` dataset:*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Summary after clipping dataset:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------- Summary statistics -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dtype</th>\n",
       "      <th>count</th>\n",
       "      <th>min</th>\n",
       "      <th>mean</th>\n",
       "      <th>median</th>\n",
       "      <th>max</th>\n",
       "      <th>std</th>\n",
       "      <th>skew</th>\n",
       "      <th>null_sum</th>\n",
       "      <th>null_%</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>device</th>\n",
       "      <td>category</td>\n",
       "      <td>8963</td>\n",
       "      <td>Android</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>iPhone</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>first_date</th>\n",
       "      <td>datetime64[ns]</td>\n",
       "      <td>8963</td>\n",
       "      <td>2020-12-07 00:00:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2020-12-21 00:00:00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <td>category</td>\n",
       "      <td>8963</td>\n",
       "      <td>EU</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>EU</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user_id</th>\n",
       "      <td>object</td>\n",
       "      <td>8963</td>\n",
       "      <td>0002CE61FF2C4011</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FFF58BC33966EB51</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     dtype  count                  min  mean  median  \\\n",
       "device            category   8963              Android   NaN     NaN   \n",
       "first_date  datetime64[ns]   8963  2020-12-07 00:00:00   NaN     NaN   \n",
       "region            category   8963                   EU   NaN     NaN   \n",
       "user_id             object   8963     0002CE61FF2C4011   NaN     NaN   \n",
       "\n",
       "                            max  std  skew  null_sum  null_%  \n",
       "device                   iPhone  NaN   NaN         0     0.0  \n",
       "first_date  2020-12-21 00:00:00  NaN   NaN         0     0.0  \n",
       "region                       EU  NaN   NaN         0     0.0  \n",
       "user_id        FFF58BC33966EB51  NaN   NaN         0     0.0  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset head -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>first_date</th>\n",
       "      <th>region</th>\n",
       "      <th>device</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2E1BF1D4C37EA01F</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>50734A22C0C63768</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>5BE017E9C8CC42F8</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>96F27A054B191457</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>E6AF85675078215D</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             user_id first_date region   device\n",
       "2   2E1BF1D4C37EA01F 2020-12-07     EU       PC\n",
       "3   50734A22C0C63768 2020-12-07     EU   iPhone\n",
       "14  5BE017E9C8CC42F8 2020-12-07     EU  Android\n",
       "18  96F27A054B191457 2020-12-07     EU   iPhone\n",
       "21  E6AF85675078215D 2020-12-07     EU  Android"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset sample -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>first_date</th>\n",
       "      <th>region</th>\n",
       "      <th>device</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8774</th>\n",
       "      <td>86779835A6AC404C</td>\n",
       "      <td>2020-12-14</td>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6614</th>\n",
       "      <td>83488534DA56F5EF</td>\n",
       "      <td>2020-12-14</td>\n",
       "      <td>EU</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14331</th>\n",
       "      <td>87C896A8F2D516CC</td>\n",
       "      <td>2020-12-21</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3092</th>\n",
       "      <td>16DEE745B1F754A2</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16669</th>\n",
       "      <td>98ECBFAF889E884E</td>\n",
       "      <td>2020-12-21</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                user_id first_date region   device\n",
       "8774   86779835A6AC404C 2020-12-14     EU       PC\n",
       "6614   83488534DA56F5EF 2020-12-14     EU   iPhone\n",
       "14331  87C896A8F2D516CC 2020-12-21     EU  Android\n",
       "3092   16DEE745B1F754A2 2020-12-07     EU       PC\n",
       "16669  98ECBFAF889E884E 2020-12-21     EU  Android"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- The number of rows, columns -----------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(8963, 4)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eda_df(df_new_eu_users)\n",
    "print('\\n----------------- Dataset head -----------------')\n",
    "df_new_eu_users.head()\n",
    "print('\\n----------------- Dataset sample -----------------')\n",
    "df_new_eu_users.sample(5)\n",
    "print('\\n----------------- The number of rows, columns -----------------')\n",
    "df_new_eu_users.shape "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9965, 3)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_eu_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*The dataset of new EU users who signed up in the online store from December 7 to 21, 2020 contains fewer unique users than the dataset containing test participants:*\n",
    "* dataset `df_new_eu_users`: 8963 users\n",
    "* dataset ` df_eu_test`: 9965 users\n",
    "\n",
    "*I can assume that some users took part in A / B testing without prior registration in the online store, for example, using the 'Buy Now' button, etc. We can check this version later.*\n",
    "\n",
    "*So let's continue with our new EU users.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first_date</th>\n",
       "      <th>n_users</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>868</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020-12-08</td>\n",
       "      <td>515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020-12-09</td>\n",
       "      <td>350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-12-10</td>\n",
       "      <td>464</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-12-11</td>\n",
       "      <td>389</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2020-12-12</td>\n",
       "      <td>694</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2020-12-13</td>\n",
       "      <td>738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2020-12-14</td>\n",
       "      <td>886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2020-12-15</td>\n",
       "      <td>486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2020-12-16</td>\n",
       "      <td>324</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>2020-12-17</td>\n",
       "      <td>491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>2020-12-18</td>\n",
       "      <td>530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>2020-12-19</td>\n",
       "      <td>538</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>2020-12-20</td>\n",
       "      <td>689</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>2020-12-21</td>\n",
       "      <td>1001</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   first_date  n_users\n",
       "0  2020-12-07      868\n",
       "1  2020-12-08      515\n",
       "2  2020-12-09      350\n",
       "3  2020-12-10      464\n",
       "4  2020-12-11      389\n",
       "5  2020-12-12      694\n",
       "6  2020-12-13      738\n",
       "7  2020-12-14      886\n",
       "8  2020-12-15      486\n",
       "9  2020-12-16      324\n",
       "10 2020-12-17      491\n",
       "11 2020-12-18      530\n",
       "12 2020-12-19      538\n",
       "13 2020-12-20      689\n",
       "14 2020-12-21     1001"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"450\" style=\"\" viewBox=\"0 0 700 450\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"450\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-ddd327\"><g class=\"clips\"><clipPath id=\"clipddd327xyplot\" class=\"plotclip\"><rect width=\"540\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clipddd327x\"><rect x=\"80\" y=\"0\" width=\"540\" height=\"450\"/></clipPath><clipPath class=\"axesclip\" id=\"clipddd327y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clipddd327xy\"><rect x=\"80\" y=\"100\" width=\"540\" height=\"270\"/></clipPath></g><g class=\"gradients\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(118.57,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(195.70999999999998,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(272.86,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(350,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(427.14,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(504.29,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(581.4300000000001,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,365.11)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,329.22)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,293.33000000000004)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,257.43)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,221.54000000000002)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,185.65)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,149.75)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,113.86)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"/><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80, 100)\" clip-path=\"url('#clipddd327xyplot')\"><g class=\"scatterlayer mlayer\"><g class=\"trace scatter tracea887bc\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,61.24L38.57,187.94L77.14,247.17L115.71,206.25L154.29,233.17L192.86,123.69L231.43,107.9L270,54.78L308.57,198.35L347.14,256.5L385.71,196.56L424.29,182.56L462.86,179.69L501.43,125.49L540,13.5\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(118.57,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\"><tspan class=\"line\" dy=\"0em\" x=\"0\" y=\"383\">Dec 8</tspan><tspan class=\"line\" dy=\"1.3em\" x=\"0\" y=\"383\">2020</tspan></text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(195.70999999999998,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 10</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(272.86,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 12</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(350,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 14</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(427.14,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 16</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(504.29,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 18</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(581.4300000000001,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 20</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,365.11)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">300</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,329.22)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">400</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,293.33000000000004)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">500</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,257.43)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">600</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,221.54000000000002)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">700</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,185.65)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">800</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,149.75)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">900</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,113.86)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">1000</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-ddd327\"><g class=\"clips\"/></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"350\" y=\"50\" text-anchor=\"middle\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">The daily number of newly signed users.</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"350\" y=\"422\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Session Date</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,42,235)\" x=\"42\" y=\"235\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Number of Users</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tmp_users_by_date = df_new_eu_users.groupby('first_date')['user_id'].agg([\n",
    "    ('n_users', 'count')]).reset_index()\n",
    "\n",
    "tmp_users_by_date\n",
    "\n",
    "fig = px.line(tmp_users_by_date,\n",
    "              x=\"first_date\",\n",
    "              y=\"n_users\",\n",
    "              title='The daily number of newly signed users.',\n",
    "              labels={\n",
    "                  'n_users': 'Number of Users',\n",
    "                  'first_date': 'Session Date'\n",
    "              })\n",
    "\n",
    "fig.show('svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* However, in fact, EU users make up 82% of our dataset.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>device</th>\n",
       "      <th>n_users</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Android</td>\n",
       "      <td>4022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>PC</td>\n",
       "      <td>2281</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>iPhone</td>\n",
       "      <td>1730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mac</td>\n",
       "      <td>930</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    device  n_users\n",
       "0  Android     4022\n",
       "1       PC     2281\n",
       "2   iPhone     1730\n",
       "3      Mac      930"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"450\" style=\"\" viewBox=\"0 0 700 450\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"450\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-9f24af\"><g class=\"clips\"/><g class=\"gradients\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"/><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"><g class=\"trace\" stroke-linejoin=\"round\" style=\"opacity: 1;\"><g class=\"slice\"><path class=\"surface\" d=\"M332.5,235l0,-135a135,135 0 0 1 42.73751957701359,263.0566453582335Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(399.5603691243971,228.9019066011111)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\">44.9%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M332.5,235l-134.94626470060476,3.8086274895704118a135,135 0 0 1 134.94626470060476,-138.8086274895704Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(276.124965865862,184.99048136036387)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">25.4%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M332.5,235l-43.72370318570655,127.72328597295105a135,135 0 0 1 -91.22256151489822,-123.91465848338063Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(44, 160, 44); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(263.248638654345,290.777821464794)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\">19.3%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M332.5,235l42.737519577013565,128.05664535823348a135,135 0 0 1 -86.46122276272011,-0.33335938528243503Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(214, 39, 40); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(332.1057492369072,342.05109239142934)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\">10.4%</text></g></g></g></g><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-9f24af\"><g class=\"clips\"/><clipPath id=\"legend9f24af\"><rect width=\"93\" height=\"86\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(595.1, 100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\" width=\"93\" height=\"86\" x=\"0\" y=\"0\"/><g class=\"scrollbox\" transform=\"translate(0, 0)\" clip-path=\"url('#legend9f24af')\"><g class=\"groups\"><g class=\"traces\" style=\"opacity: 1;\" transform=\"translate(0, 14.5)\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Android</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"87.265625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" style=\"opacity: 1;\" transform=\"translate(0, 33.5)\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">PC</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"87.265625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" style=\"opacity: 1;\" transform=\"translate(0, 52.5)\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">iPhone</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(44, 160, 44); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"87.265625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" style=\"opacity: 1;\" transform=\"translate(0, 71.5)\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Mac</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(214, 39, 40); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"87.265625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\" x=\"0\" y=\"0\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"350\" y=\"50\" text-anchor=\"middle\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">The number of newly signed users by device.</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tmp_by_device = df_new_eu_users.groupby('device')['user_id'].agg([\n",
    "    ('n_users', 'count')\n",
    "]).sort_values(by='n_users', ascending=False).reset_index()\n",
    "\n",
    "tmp_by_device\n",
    "\n",
    "fig = px.pie(tmp_by_device,\n",
    "             values='n_users',\n",
    "             names='device',\n",
    "#              color='region',\n",
    "             title='The number of newly signed users by device.')\n",
    "\n",
    "fig.show('svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Conclusion:\n",
    "* After preprocessing we have the `df_new_eu_users` dataset 8963 unique entries.\n",
    "* All new users in the dataset are from the EU who signed up in the online store from December 7 to 21, 2020.\n",
    "* New EU user registrations for 14 days of testing varied greatly:\n",
    "    - The maximum number of new users came on December 21st - 1001 new users, \n",
    "    - The minimum - on December 16th -  324 new users,   \n",
    "    - The number of new user registrations depends on the day of week.\n",
    "* They utilize 4 types of devices: PC, Android, iPhone, Mac:\n",
    "    - Almost half of the new users are Android users, 44.9%. \n",
    "    - The smallest number of users came using the Mac, 10.4%.\n",
    "* There are no null values and duplicates in the dataset.\n",
    "* **According to the assignment, the estimated number of test participants must be at least 6000. We have 8963 users, which meets the conditions of testing.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The `events` dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Downloading:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_events = pd.read_csv(os.path.join(source_dir, events),\n",
    "                        parse_dates=['event_dt'],\n",
    "                        na_values=missing_values,\n",
    "                        dtype={'event_name': 'category'})\n",
    "\n",
    "# saving the original data set for comparing changes to be made\n",
    "orig_df_events = df_events.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Overview and descriptive statistic:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- A concise summary of the DataFrame -----------------\n",
      "\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 423761 entries, 0 to 423760\n",
      "Data columns (total 4 columns):\n",
      " #   Column      Non-Null Count   Dtype         \n",
      "---  ------      --------------   -----         \n",
      " 0   user_id     423761 non-null  object        \n",
      " 1   event_dt    423761 non-null  datetime64[ns]\n",
      " 2   event_name  423761 non-null  category      \n",
      " 3   details     60314 non-null   float64       \n",
      "dtypes: category(1), datetime64[ns](1), float64(1), object(1)\n",
      "memory usage: 10.1+ MB\n",
      "\n",
      "----------------- Dataset head -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>event_dt</th>\n",
       "      <th>event_name</th>\n",
       "      <th>details</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>E1BDDCE0DAFA2679</td>\n",
       "      <td>2020-12-07 20:22:03</td>\n",
       "      <td>purchase</td>\n",
       "      <td>99.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7B6452F081F49504</td>\n",
       "      <td>2020-12-07 09:22:53</td>\n",
       "      <td>purchase</td>\n",
       "      <td>9.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9CD9F34546DF254C</td>\n",
       "      <td>2020-12-07 12:59:29</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>96F27A054B191457</td>\n",
       "      <td>2020-12-07 04:02:40</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1FD7660FDF94CA1F</td>\n",
       "      <td>2020-12-07 10:15:09</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            user_id            event_dt event_name  details\n",
       "0  E1BDDCE0DAFA2679 2020-12-07 20:22:03   purchase    99.99\n",
       "1  7B6452F081F49504 2020-12-07 09:22:53   purchase     9.99\n",
       "2  9CD9F34546DF254C 2020-12-07 12:59:29   purchase     4.99\n",
       "3  96F27A054B191457 2020-12-07 04:02:40   purchase     4.99\n",
       "4  1FD7660FDF94CA1F 2020-12-07 10:15:09   purchase     4.99"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset tail -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>event_dt</th>\n",
       "      <th>event_name</th>\n",
       "      <th>details</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>423756</th>\n",
       "      <td>245E85F65C358E08</td>\n",
       "      <td>2020-12-30 19:35:55</td>\n",
       "      <td>login</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>423757</th>\n",
       "      <td>9385A108F5A0A7A7</td>\n",
       "      <td>2020-12-30 10:54:15</td>\n",
       "      <td>login</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>423758</th>\n",
       "      <td>DB650B7559AC6EAC</td>\n",
       "      <td>2020-12-30 10:59:09</td>\n",
       "      <td>login</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>423759</th>\n",
       "      <td>F80C9BDDEA02E53C</td>\n",
       "      <td>2020-12-30 09:53:39</td>\n",
       "      <td>login</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>423760</th>\n",
       "      <td>7AEC61159B672CC5</td>\n",
       "      <td>2020-12-30 11:36:13</td>\n",
       "      <td>login</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id            event_dt event_name  details\n",
       "423756  245E85F65C358E08 2020-12-30 19:35:55      login      NaN\n",
       "423757  9385A108F5A0A7A7 2020-12-30 10:54:15      login      NaN\n",
       "423758  DB650B7559AC6EAC 2020-12-30 10:59:09      login      NaN\n",
       "423759  F80C9BDDEA02E53C 2020-12-30 09:53:39      login      NaN\n",
       "423760  7AEC61159B672CC5 2020-12-30 11:36:13      login      NaN"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset sample -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>event_dt</th>\n",
       "      <th>event_name</th>\n",
       "      <th>details</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>394448</th>\n",
       "      <td>FCE02BA36FAACBDE</td>\n",
       "      <td>2020-12-23 22:08:16</td>\n",
       "      <td>login</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81159</th>\n",
       "      <td>2BA840498F1FCA75</td>\n",
       "      <td>2020-12-15 15:56:30</td>\n",
       "      <td>product_cart</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>410557</th>\n",
       "      <td>87891D2E95B32B53</td>\n",
       "      <td>2020-12-27 02:29:09</td>\n",
       "      <td>login</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>249691</th>\n",
       "      <td>907A0A000E3F92D5</td>\n",
       "      <td>2020-12-08 08:54:10</td>\n",
       "      <td>login</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1523</th>\n",
       "      <td>2EBF2E73A8630674</td>\n",
       "      <td>2020-12-07 19:11:09</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id            event_dt    event_name  details\n",
       "394448  FCE02BA36FAACBDE 2020-12-23 22:08:16         login      NaN\n",
       "81159   2BA840498F1FCA75 2020-12-15 15:56:30  product_cart      NaN\n",
       "410557  87891D2E95B32B53 2020-12-27 02:29:09         login      NaN\n",
       "249691  907A0A000E3F92D5 2020-12-08 08:54:10         login      NaN\n",
       "1523    2EBF2E73A8630674 2020-12-07 19:11:09      purchase     4.99"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- The number of rows, columns -----------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(423761, 4)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('\\n----------------- A concise summary of the DataFrame -----------------\\n')\n",
    "df_events.info()\n",
    "print('\\n----------------- Dataset head -----------------')\n",
    "df_events.head()\n",
    "print('\\n----------------- Dataset tail -----------------')\n",
    "df_events.tail()\n",
    "print('\\n----------------- Dataset sample -----------------')\n",
    "df_events.sample(5)\n",
    "print('\\n----------------- The number of rows, columns -----------------')\n",
    "df_events.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Count unique values in the columns:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "user_id        58703\n",
       "event_dt      257138\n",
       "event_name         4\n",
       "details            5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_events.apply(lambda x: len(x.unique()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "event_name : [purchase, product_cart, product_page, login]\n",
      "Categories (4, object): [purchase, product_cart, product_page, login]\n",
      "details : [ 99.99   9.99   4.99 499.99    nan]\n"
     ]
    }
   ],
   "source": [
    "for i in df_events[['event_name', 'details']]:\n",
    "    print(i, ':', df_events[i].unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Let's explore Nan values in the `details` column:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "363447"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "85.77"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "details_null = df_events.details.isna().sum()\n",
    "details_null\n",
    "(details_null / df_events.shape[0] * 100).round(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*As we can see, our `event` dataset has 85.77% NaN values in the `details` column.\n",
    "<br>I think this happened because not all of our users did reach the `purchase` event. We can check this version)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>event_dt</th>\n",
       "      <th>event_name</th>\n",
       "      <th>details</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [user_id, event_dt, event_name, details]\n",
       "Index: []"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_events[(df_events.event_name == 'purchase') & df_events.details.isna()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*There is no 'purchase' event with NaN value in the 'details' column. But previous steps of funnel have Nan's.*\n",
    "\n",
    "*I consider that we can fill Nan's with 0.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "user_id       0\n",
       "event_dt      0\n",
       "event_name    0\n",
       "details       0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# filling Nan\n",
    "df_events.details.fillna(0, inplace=True)\n",
    "\n",
    "# Verifying the result\n",
    "df_events.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Checking duplicates:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_events.duplicated().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count                  423761\n",
       "unique                 257138\n",
       "top       2020-12-23 02:37:24\n",
       "freq                       10\n",
       "first     2020-12-07 00:00:33\n",
       "last      2020-12-30 23:36:33\n",
       "Name: event_dt, dtype: object"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_events.event_dt.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Preliminary overview:\n",
    "* The `events` dataset contains 423761 records from December 7 to December 30, 2020.\n",
    "* The funnel consists 4 events: login, product_page, product_cart, and purchase. \n",
    "<br>However, in accordance with our task, we are interested in a funnel of these 3 steps: `product_page → product_card → purchase`. \n",
    "* There are no duplicates.\n",
    "* Nan values of the variable `details` were filled with 0."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*To prepare the dataset for further exploration, we need to perform the same data transformation as for the new user dataset, namely:*\n",
    "* First, get the data for our testing from December 7 to December 21;\n",
    "* Second, we need to retrieve the events that were executed by our test participants.*\n",
    "\n",
    "*Let's create the `df_test_events` for our goals and explore it:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------- Summary statistics -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dtype</th>\n",
       "      <th>count</th>\n",
       "      <th>min</th>\n",
       "      <th>mean</th>\n",
       "      <th>median</th>\n",
       "      <th>max</th>\n",
       "      <th>std</th>\n",
       "      <th>skew</th>\n",
       "      <th>null_sum</th>\n",
       "      <th>null_%</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>details</th>\n",
       "      <td>float64</td>\n",
       "      <td>51848</td>\n",
       "      <td>0</td>\n",
       "      <td>3.562788</td>\n",
       "      <td>0.0</td>\n",
       "      <td>499.99</td>\n",
       "      <td>29.740594</td>\n",
       "      <td>14.689667</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>event_dt</th>\n",
       "      <td>datetime64[ns]</td>\n",
       "      <td>51848</td>\n",
       "      <td>2020-12-07 00:02:48</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2020-12-21 23:58:37</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>event_name</th>\n",
       "      <td>category</td>\n",
       "      <td>51848</td>\n",
       "      <td>login</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>purchase</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user_id</th>\n",
       "      <td>object</td>\n",
       "      <td>51848</td>\n",
       "      <td>0002CE61FF2C4011</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FFF58BC33966EB51</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     dtype  count                  min      mean  median  \\\n",
       "details            float64  51848                    0  3.562788     0.0   \n",
       "event_dt    datetime64[ns]  51848  2020-12-07 00:02:48       NaN     NaN   \n",
       "event_name        category  51848                login       NaN     NaN   \n",
       "user_id             object  51848     0002CE61FF2C4011       NaN     NaN   \n",
       "\n",
       "                            max        std       skew  null_sum  null_%  \n",
       "details                  499.99  29.740594  14.689667         0     0.0  \n",
       "event_dt    2020-12-21 23:58:37        NaN        NaN         0     0.0  \n",
       "event_name             purchase        NaN        NaN         0     0.0  \n",
       "user_id        FFF58BC33966EB51        NaN        NaN         0     0.0  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- Dataset sample -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>event_dt</th>\n",
       "      <th>event_name</th>\n",
       "      <th>details</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>288774</th>\n",
       "      <td>51D0247130DA073F</td>\n",
       "      <td>2020-12-14 00:24:05</td>\n",
       "      <td>login</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>327228</th>\n",
       "      <td>ED0B128C136A33BB</td>\n",
       "      <td>2020-12-18 04:43:23</td>\n",
       "      <td>login</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>127741</th>\n",
       "      <td>59E66153942FF602</td>\n",
       "      <td>2020-12-09 00:37:11</td>\n",
       "      <td>product_page</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>274098</th>\n",
       "      <td>C89BE207AFEE66B8</td>\n",
       "      <td>2020-12-12 03:43:10</td>\n",
       "      <td>login</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>315239</th>\n",
       "      <td>E935EF793A7E4C06</td>\n",
       "      <td>2020-12-16 11:17:59</td>\n",
       "      <td>login</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 user_id            event_dt    event_name  details\n",
       "288774  51D0247130DA073F 2020-12-14 00:24:05         login      0.0\n",
       "327228  ED0B128C136A33BB 2020-12-18 04:43:23         login      0.0\n",
       "127741  59E66153942FF602 2020-12-09 00:37:11  product_page      0.0\n",
       "274098  C89BE207AFEE66B8 2020-12-12 03:43:10         login      0.0\n",
       "315239  E935EF793A7E4C06 2020-12-16 11:17:59         login      0.0"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test_events = df_events[(df_events['user_id'].isin(lst_eu_test_users)) & (df_events['event_dt'] < \"2020-12-22\")]\n",
    "\n",
    "eda_df(df_test_events)\n",
    "print('\\n----------------- Dataset sample -----------------')\n",
    "df_test_events.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Count unique values in the columns:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "user_id        8959\n",
       "event_dt      33338\n",
       "event_name        4\n",
       "details           5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>purchase</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>product_cart</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>product_page</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>login</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     event_name\n",
       "0      purchase\n",
       "1  product_cart\n",
       "2  product_page\n",
       "3         login"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test_events.apply(lambda x: len(x.unique()))\n",
    "\n",
    "df_test_events[['event_name']].apply(lambda x: x.unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Verifying that there are no duplicate values in the dataset:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test_events.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*First look at the number of events per user:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    8959.000000\n",
       "mean        5.787253\n",
       "std         3.727047\n",
       "min         1.000000\n",
       "25%         3.000000\n",
       "50%         6.000000\n",
       "75%         8.000000\n",
       "max        24.000000\n",
       "Name: n_events, dtype: float64"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "0    6\n",
       "dtype: int64"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp_events = df_test_events.groupby('user_id')['event_dt'].agg([\n",
    "    ('n_events', 'count')\n",
    "]).sort_values(by='n_events', ascending=False).reset_index()\n",
    "\n",
    "tmp_events.n_events.describe()\n",
    "\n",
    "# most frequently number of events\n",
    "tmp_events.n_events.mode()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Conclusion:\n",
    "* The `df_test_events` dataset contains 51848 events.\n",
    "* The dataset includes data that was collected from December 7 to December 21, 2020.\n",
    "* All events presented in the dataset were performed by test takers.\n",
    "* Each participant completed at least 1 event. Maximum events performed by user is 24.\n",
    "* An average number of events per user is 6.\n",
    "* Most often the number of events per user is 6, which coincides with the average number of events."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Merging datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*After preprocessing our datasets, we can combine them for deeper exploratory analysis.\n",
    "<br>But first, a small reminder of which ones we will use and what they look like :)* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "-----------------  Table containing test participants   -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>group</th>\n",
       "      <th>ab_test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6701</th>\n",
       "      <td>D4E530F6595A05A3</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6703</th>\n",
       "      <td>6BCB0F33D3BAB8C2</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6707</th>\n",
       "      <td>2D2E7AA539CF348F</td>\n",
       "      <td>B</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6708</th>\n",
       "      <td>4BA448BCE1343C6F</td>\n",
       "      <td>A</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6709</th>\n",
       "      <td>76B6CDF2A8B1DBFC</td>\n",
       "      <td>B</td>\n",
       "      <td>interface_eu_test</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               user_id group            ab_test\n",
       "6701  D4E530F6595A05A3     A  interface_eu_test\n",
       "6703  6BCB0F33D3BAB8C2     A  interface_eu_test\n",
       "6707  2D2E7AA539CF348F     B  interface_eu_test\n",
       "6708  4BA448BCE1343C6F     A  interface_eu_test\n",
       "6709  76B6CDF2A8B1DBFC     B  interface_eu_test"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(9965, 3)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------- New EU users who signed up in the online store from December 7 to 21, 2020 -----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>first_date</th>\n",
       "      <th>region</th>\n",
       "      <th>device</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2E1BF1D4C37EA01F</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>50734A22C0C63768</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>5BE017E9C8CC42F8</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>96F27A054B191457</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>E6AF85675078215D</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>EU</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             user_id first_date region   device\n",
       "2   2E1BF1D4C37EA01F 2020-12-07     EU       PC\n",
       "3   50734A22C0C63768 2020-12-07     EU   iPhone\n",
       "14  5BE017E9C8CC42F8 2020-12-07     EU  Android\n",
       "18  96F27A054B191457 2020-12-07     EU   iPhone\n",
       "21  E6AF85675078215D 2020-12-07     EU  Android"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(8963, 4)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----------- All the events of the new users (test participants) within the period from December 7 to 21, 2020 ------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>event_dt</th>\n",
       "      <th>event_name</th>\n",
       "      <th>details</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>96F27A054B191457</td>\n",
       "      <td>2020-12-07 04:02:40</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>A92195E3CFB83DBD</td>\n",
       "      <td>2020-12-07 00:32:07</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>354D653172FF2A2D</td>\n",
       "      <td>2020-12-07 15:45:11</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>7FCD34F47C13A9AC</td>\n",
       "      <td>2020-12-07 22:06:13</td>\n",
       "      <td>purchase</td>\n",
       "      <td>9.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0313C457F07C339E</td>\n",
       "      <td>2020-12-07 13:10:48</td>\n",
       "      <td>purchase</td>\n",
       "      <td>9.99</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             user_id            event_dt event_name  details\n",
       "3   96F27A054B191457 2020-12-07 04:02:40   purchase     4.99\n",
       "9   A92195E3CFB83DBD 2020-12-07 00:32:07   purchase     4.99\n",
       "11  354D653172FF2A2D 2020-12-07 15:45:11   purchase     4.99\n",
       "12  7FCD34F47C13A9AC 2020-12-07 22:06:13   purchase     9.99\n",
       "13  0313C457F07C339E 2020-12-07 13:10:48   purchase     9.99"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(51848, 4)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('\\n-----------------  Table containing test participants   -----------------')\n",
    "df_eu_test.head()\n",
    "df_eu_test.shape\n",
    "\n",
    "print('\\n----------------- New EU users who signed up in the online store from December 7 to 21, 2020 -----------------')\n",
    "df_new_eu_users.head()\n",
    "df_new_eu_users.shape\n",
    "\n",
    "print('\\n----------- All the events of the new users (test participants) within the period from December 7 to 21, 2020 ------------')\n",
    "df_test_events.head()\n",
    "df_test_events.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Combining the datasets:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>event_dt</th>\n",
       "      <th>event_name</th>\n",
       "      <th>details</th>\n",
       "      <th>group</th>\n",
       "      <th>first_date</th>\n",
       "      <th>device</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>96F27A054B191457</td>\n",
       "      <td>2020-12-07 04:02:40</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "      <td>B</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>iPhone</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A92195E3CFB83DBD</td>\n",
       "      <td>2020-12-07 00:32:07</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "      <td>A</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>Android</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>354D653172FF2A2D</td>\n",
       "      <td>2020-12-07 15:45:11</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "      <td>A</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>Mac</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7FCD34F47C13A9AC</td>\n",
       "      <td>2020-12-07 22:06:13</td>\n",
       "      <td>purchase</td>\n",
       "      <td>9.99</td>\n",
       "      <td>B</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0313C457F07C339E</td>\n",
       "      <td>2020-12-07 13:10:48</td>\n",
       "      <td>purchase</td>\n",
       "      <td>9.99</td>\n",
       "      <td>A</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>Mac</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            user_id            event_dt event_name  details group first_date  \\\n",
       "0  96F27A054B191457 2020-12-07 04:02:40   purchase     4.99     B 2020-12-07   \n",
       "1  A92195E3CFB83DBD 2020-12-07 00:32:07   purchase     4.99     A 2020-12-07   \n",
       "2  354D653172FF2A2D 2020-12-07 15:45:11   purchase     4.99     A 2020-12-07   \n",
       "3  7FCD34F47C13A9AC 2020-12-07 22:06:13   purchase     9.99     B 2020-12-07   \n",
       "4  0313C457F07C339E 2020-12-07 13:10:48   purchase     9.99     A 2020-12-07   \n",
       "\n",
       "    device  \n",
       "0   iPhone  \n",
       "1  Android  \n",
       "2      Mac  \n",
       "3       PC  \n",
       "4      Mac  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_ab_testing = df_test_events.merge(df_eu_test[['user_id', 'group']],\n",
    "                                     on='user_id', how='left').merge(df_new_eu_users[[\n",
    "                                         'user_id', 'first_date', 'device'\n",
    "                                     ]], on='user_id', how='left')\n",
    "\n",
    "df_ab_testing.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "user_id       0\n",
       "event_dt      0\n",
       "event_name    0\n",
       "details       0\n",
       "group         0\n",
       "first_date    0\n",
       "device        0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking some features after merging\n",
    "df_ab_testing.isna().sum()\n",
    "df_ab_testing.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploratory Data Analysis (EDA)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Study conversion at different funnel stages."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*The main goal of our test is to make sure that the total conversion [1] on each stage in Test Group B is better than the total conversion at the same stage in Control Group A.*\n",
    "\n",
    "[1] 'Total conversion is the ratio of users at a particular stage to the number of users at the first stage.'\n",
    "\n",
    "*Let's plot the funnel and compare the groups.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1) *Calculate the number of users for each stage in group A:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_name</th>\n",
       "      <th>n_users</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>login</td>\n",
       "      <td>11247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>product_page</td>\n",
       "      <td>7515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>purchase</td>\n",
       "      <td>4031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>product_cart</td>\n",
       "      <td>3720</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     event_name  n_users\n",
       "0         login    11247\n",
       "1  product_page     7515\n",
       "2      purchase     4031\n",
       "3  product_cart     3720"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_funnel_A = df_ab_testing[df_ab_testing['group'] == \"A\"].groupby(\n",
    "    ['event_name'])['user_id'].agg([\n",
    "        ('n_users', 'count')\n",
    "    ]).sort_values(by=['n_users'], ascending=False).reset_index()\n",
    "\n",
    "df_funnel_A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2) *Calculating total conversion for each funnel stage in group A:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11247"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first_stage_A = df_funnel_A.iloc[0, 1]\n",
    "first_stage_A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_name</th>\n",
       "      <th>n_users</th>\n",
       "      <th>total_conversion</th>\n",
       "      <th>test_group</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>login</td>\n",
       "      <td>11247</td>\n",
       "      <td>100.00</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>product_page</td>\n",
       "      <td>7515</td>\n",
       "      <td>66.82</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>purchase</td>\n",
       "      <td>4031</td>\n",
       "      <td>35.84</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>product_cart</td>\n",
       "      <td>3720</td>\n",
       "      <td>33.08</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     event_name  n_users  total_conversion test_group\n",
       "0         login    11247            100.00          A\n",
       "1  product_page     7515             66.82          A\n",
       "2      purchase     4031             35.84          A\n",
       "3  product_cart     3720             33.08          A"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_funnel_A = df_funnel_A.assign(\n",
    "    total_conversion=lambda x: round(x.n_users / first_stage_A * 100, 2),\n",
    "    test_group='A')\n",
    "\n",
    "df_funnel_A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*The same steps for the group B:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10885"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_name</th>\n",
       "      <th>n_users</th>\n",
       "      <th>total_conversion</th>\n",
       "      <th>test_group</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>login</td>\n",
       "      <td>10885</td>\n",
       "      <td>100.00</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>product_page</td>\n",
       "      <td>7102</td>\n",
       "      <td>65.25</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>product_cart</td>\n",
       "      <td>3722</td>\n",
       "      <td>34.19</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>purchase</td>\n",
       "      <td>3626</td>\n",
       "      <td>33.31</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     event_name  n_users  total_conversion test_group\n",
       "0         login    10885            100.00          B\n",
       "1  product_page     7102             65.25          B\n",
       "2  product_cart     3722             34.19          B\n",
       "3      purchase     3626             33.31          B"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_funnel_B = df_ab_testing[df_ab_testing['group'] == \"B\"].groupby(\n",
    "    ['event_name'])['user_id'].agg([\n",
    "        ('n_users', 'count')\n",
    "    ]).sort_values(by=['n_users'], ascending=False).reset_index()\n",
    "\n",
    "# df_funnel_B\n",
    "\n",
    "first_stage_B = df_funnel_B.iloc[0, 1]\n",
    "first_stage_B\n",
    "\n",
    "df_funnel_B = df_funnel_B.assign(\n",
    "    total_conversion=lambda x: round(x.n_users / first_stage_B * 100, 2),\n",
    "    test_group='B')\n",
    "\n",
    "df_funnel_B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Concatenating test groups and plotting funnel chart:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_name</th>\n",
       "      <th>n_users</th>\n",
       "      <th>total_conversion</th>\n",
       "      <th>test_group</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>login</td>\n",
       "      <td>11247</td>\n",
       "      <td>100.00</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>product_page</td>\n",
       "      <td>7515</td>\n",
       "      <td>66.82</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>purchase</td>\n",
       "      <td>4031</td>\n",
       "      <td>35.84</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>product_cart</td>\n",
       "      <td>3720</td>\n",
       "      <td>33.08</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>login</td>\n",
       "      <td>10885</td>\n",
       "      <td>100.00</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>product_page</td>\n",
       "      <td>7102</td>\n",
       "      <td>65.25</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>product_cart</td>\n",
       "      <td>3722</td>\n",
       "      <td>34.19</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>purchase</td>\n",
       "      <td>3626</td>\n",
       "      <td>33.31</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     event_name  n_users  total_conversion test_group\n",
       "0         login    11247            100.00          A\n",
       "1  product_page     7515             66.82          A\n",
       "2      purchase     4031             35.84          A\n",
       "3  product_cart     3720             33.08          A\n",
       "0         login    10885            100.00          B\n",
       "1  product_page     7102             65.25          B\n",
       "2  product_cart     3722             34.19          B\n",
       "3      purchase     3626             33.31          B"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"450\" style=\"\" viewBox=\"0 0 700 450\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"450\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-0213b9\"><g class=\"clips\"><clipPath id=\"clip0213b9xyplot\" class=\"plotclip\"><rect width=\"518\" height=\"310\"/></clipPath><clipPath class=\"axesclip\" id=\"clip0213b9x\"><rect x=\"80\" y=\"0\" width=\"518\" height=\"450\"/></clipPath><clipPath class=\"axesclip\" id=\"clip0213b9y\"><rect x=\"0\" y=\"60\" width=\"700\" height=\"310\"/></clipPath><clipPath class=\"axesclip\" id=\"clip0213b9xy\"><rect x=\"80\" y=\"60\" width=\"518\" height=\"310\"/></clipPath></g><g class=\"gradients\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"/><g class=\"y\"/></g><g class=\"zerolinelayer\"/><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80, 60)\" clip-path=\"url('#clip0213b9xyplot')\"><g class=\"funnellayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"regions\"><g class=\"region\"><path d=\"M25.9,69.75L105.07,85.25H260.83L259,69.75Z\" style=\"stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 0.5;\"/></g><g class=\"region\"><path d=\"M105.07,147.25L178.41,162.75H261.95L260.83,147.25Z\" style=\"stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 0.5;\"/></g><g class=\"region\"><path d=\"M178.41,224.75L180.6,240.25H257.71L261.95,224.75Z\" style=\"stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 0.5;\"/></g><g class=\"region\"><path d=\"M0,0Z\" style=\"stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 0.5;\"/></g></g><g class=\"points\"><g class=\"point\"><path d=\"M25.9,7.75V69.75H259V7.75Z\" style=\"vector-effect: non-scaling-stroke; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 0px; opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\" transform=\"translate(142.45,43.546875)\">100</text></g><g class=\"point\"><path d=\"M105.07,85.25V147.25H260.83V85.25Z\" style=\"vector-effect: non-scaling-stroke; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 0px; opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\" transform=\"translate(182.95,121.046875)\">66.82</text></g><g class=\"point\"><path d=\"M178.41,162.75V224.75H261.95V162.75Z\" style=\"vector-effect: non-scaling-stroke; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 0px; opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\" transform=\"translate(220.0003125,198.546875)\">35.84</text></g><g class=\"point\"><path d=\"M180.6,240.25V302.25H257.71V240.25Z\" style=\"vector-effect: non-scaling-stroke; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 0px; opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\" transform=\"translate(219.15499999999997,276.046875)\">33.08</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"regions\"><g class=\"region\"><path d=\"M259,69.75L260.83,85.25H412.93L492.1,69.75Z\" style=\"stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 0.5;\"/></g><g class=\"region\"><path d=\"M260.83,147.25L257.71,240.25H337.4L412.93,147.25Z\" style=\"stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 0.5;\"/></g><g class=\"region\"><path d=\"M257.71,302.25L261.95,162.75H339.59L337.4,302.25Z\" style=\"stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 0.5;\"/></g><g class=\"region\"><path d=\"M0,0Z\" style=\"stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 0.5;\"/></g></g><g class=\"points\"><g class=\"point\"><path d=\"M259,7.75V69.75H492.1V7.75Z\" style=\"vector-effect: non-scaling-stroke; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 0px; opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(375.55,43.546875)\">100</text></g><g class=\"point\"><path d=\"M260.83,85.25V147.25H412.93V85.25Z\" style=\"vector-effect: non-scaling-stroke; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 0px; opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(336.88,121.046875)\">65.25</text></g><g class=\"point\"><path d=\"M257.71,240.25V302.25H337.4V240.25Z\" style=\"vector-effect: non-scaling-stroke; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 0px; opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(297.55499999999995,276.046875)\">34.19</text></g><g class=\"point\"><path d=\"M261.95,162.75V224.75H339.59V162.75Z\" style=\"vector-effect: non-scaling-stroke; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 0px; opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(300.77,198.546875)\">33.31</text></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" style=\"fill: none;\" d=\"M0,0\"/><path class=\"ylines-above crisp\" style=\"fill: none;\" d=\"M0,0\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"/><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,331.25)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">product_cart</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,253.75)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">purchase</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,176.25)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">product_page</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,98.75)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">login</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-0213b9\"><g class=\"clips\"/><clipPath id=\"legend0213b9\"><rect width=\"80\" height=\"64\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(608.36, 60)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\" width=\"80\" height=\"64\" x=\"0\" y=\"0\"/><g class=\"scrollbox\" transform=\"translate(0, 0)\" clip-path=\"url('#legend0213b9')\"><text class=\"legendtitletext user-select-none\" text-anchor=\"start\" x=\"2\" y=\"15.600000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Test groups:</text><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 30.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">A</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendfunnel\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(31, 119, 180); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"49.625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 49.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">B</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendfunnel\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(255, 127, 14); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"49.625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"/><g class=\"g-ytitle\"/></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_funnel_shift = pd.concat([df_funnel_A, df_funnel_B], axis=0)\n",
    "df_funnel_shift\n",
    "\n",
    "fig = px.funnel(df_funnel_shift,\n",
    "                x='total_conversion',\n",
    "                y='event_name',\n",
    "                color='test_group',\n",
    "                labels={\n",
    "                    'event_name': ' ',\n",
    "                    'test_group': 'Test groups:'\n",
    "                })\n",
    "fig.show('svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">* According to the test, it was expected that 'At each stage of the funnel `product_page → product_cart → purchase`, there will be at least a 10% increase.\"\n",
    "* However, the funnel chart shows that the conversion in the control group A is even slightly higher than in the test group B in the first two stages.\n",
    "* Only the `purchase` stage showed 1.11% better in test group B than in group A.\n",
    "* Moreover, at `purchase` stage in group B, the total conversion is higher than at the previous stage. I assume this was due to the 'Buy Now' button, which allowed users to buy by skipping some steps."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Is the number of events per user distributed equally in the samples?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>group</th>\n",
       "      <th>n_users</th>\n",
       "      <th>n_events</th>\n",
       "      <th>n_events_per_user</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>4521</td>\n",
       "      <td>26513</td>\n",
       "      <td>5.86</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B</td>\n",
       "      <td>4438</td>\n",
       "      <td>25335</td>\n",
       "      <td>5.71</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  group  n_users  n_events  n_events_per_user\n",
       "0     A     4521     26513               5.86\n",
       "1     B     4438     25335               5.71"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"450\" style=\"\" viewBox=\"0 0 700 450\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"450\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-f54e18\"><g class=\"clips\"/><g class=\"gradients\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"/><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"><g class=\"trace\" stroke-linejoin=\"round\" style=\"opacity: 1;\"><g class=\"slice\"><path class=\"surface\" d=\"M350,235l0,-135a135,135 0 1 1 -9.627822774320414,269.65624764052455Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(417.4570172697685,242.20536437430664)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\">51.1%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M350,235l-9.627822774320448,134.65624764052455a135,135 0 0 1 9.627822774320448,-269.65624764052455Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(282.52149820994356,237.38761854109757)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">48.9%</text></g></g></g></g><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-f54e18\"><g class=\"clips\"/><clipPath id=\"legendf54e18\"><rect width=\"55\" height=\"48\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(630.8, 100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"55\" height=\"48\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"translate(0, 0)\" clip-path=\"url('#legendf54e18')\"><g class=\"groups\"><g class=\"traces\" transform=\"translate(0, 14.5)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">A</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(31, 119, 180); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"49.625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0, 33.5)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">B</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(255, 127, 14); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"49.625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"350\" y=\"50\" text-anchor=\"middle\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Distribution of the number of events per user.</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tm_events_per_user = df_ab_testing.groupby(['group']).agg({\n",
    "    'user_id': 'nunique',\n",
    "    'event_dt': 'count'\n",
    "}).rename(columns={\n",
    "    'user_id': 'n_users',\n",
    "    'event_dt': 'n_events'\n",
    "}).reset_index()\n",
    "\n",
    "tm_events_per_user['n_events_per_user'] = round(tm_events_per_user.n_events / tm_events_per_user.n_users, 2)\n",
    "tm_events_per_user\n",
    "\n",
    "fig = px.pie(\n",
    "    tm_events_per_user,\n",
    "    values='n_events',\n",
    "    names='group',\n",
    "    #              color='region',\n",
    "    title='Distribution of the number of events per user.')\n",
    "\n",
    "fig.show('svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">*As we can see, the share of users and events is close to 50/50.\n",
    "<br>Thus, we can claim that the number of events per user distributed equally in the samples*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Are there users who enter both samples?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(\n",
    "    df_ab_testing.groupby(['user_id']).agg({\n",
    "        'group': 'nunique'\n",
    "    }).reset_index().query('group == 2'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">*There is no users who enter both samples.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How is the number of events distributed by days?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*To answer this question, let's create an additional column `event-date`:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>event_dt</th>\n",
       "      <th>event_name</th>\n",
       "      <th>details</th>\n",
       "      <th>group</th>\n",
       "      <th>first_date</th>\n",
       "      <th>device</th>\n",
       "      <th>event_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>96F27A054B191457</td>\n",
       "      <td>2020-12-07 04:02:40</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "      <td>B</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>iPhone</td>\n",
       "      <td>2020-12-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A92195E3CFB83DBD</td>\n",
       "      <td>2020-12-07 00:32:07</td>\n",
       "      <td>purchase</td>\n",
       "      <td>4.99</td>\n",
       "      <td>A</td>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>Android</td>\n",
       "      <td>2020-12-07</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            user_id            event_dt event_name  details group first_date  \\\n",
       "0  96F27A054B191457 2020-12-07 04:02:40   purchase     4.99     B 2020-12-07   \n",
       "1  A92195E3CFB83DBD 2020-12-07 00:32:07   purchase     4.99     A 2020-12-07   \n",
       "\n",
       "    device event_date  \n",
       "0   iPhone 2020-12-07  \n",
       "1  Android 2020-12-07  "
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_ab_testing['event_date'] = get_day(df_ab_testing, 'event_dt')\n",
    "df_ab_testing.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">*Number of total daily events:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_date</th>\n",
       "      <th>n_events</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>1883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020-12-08</td>\n",
       "      <td>2168</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020-12-09</td>\n",
       "      <td>2156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-12-10</td>\n",
       "      <td>2399</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-12-11</td>\n",
       "      <td>2465</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  event_date  n_events\n",
       "0 2020-12-07      1883\n",
       "1 2020-12-08      2168\n",
       "2 2020-12-09      2156\n",
       "3 2020-12-10      2399\n",
       "4 2020-12-11      2465"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"450\" style=\"\" viewBox=\"0 0 700 450\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"450\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-ccf1f8\"><g class=\"clips\"><clipPath id=\"clipccf1f8xyplot\" class=\"plotclip\"><rect width=\"540\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clipccf1f8x\"><rect x=\"80\" y=\"0\" width=\"540\" height=\"450\"/></clipPath><clipPath class=\"axesclip\" id=\"clipccf1f8y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clipccf1f8xy\"><rect x=\"80\" y=\"100\" width=\"540\" height=\"270\"/></clipPath></g><g class=\"gradients\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(118.57,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(195.70999999999998,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(272.86,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(350,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(427.14,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(504.29,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(581.4300000000001,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,348.66999999999996)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,281.78999999999996)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,214.9)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,148.01)\" d=\"M80,0h540\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"/><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80, 100)\" clip-path=\"url('#clipccf1f8xyplot')\"><g class=\"scatterlayer mlayer\"><g class=\"trace scatter trace2e46a0\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,256.5L38.57,237.44L77.14,238.24L115.71,221.99L154.29,217.57L192.86,157.71L231.43,125.74L270,83.6L308.57,113.23L347.14,137.44L385.71,134.77L424.29,126.67L462.86,117.71L501.43,86.67L540,13.5\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(118.57,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\"><tspan class=\"line\" dy=\"0em\" x=\"0\" y=\"383\">Dec 8</tspan><tspan class=\"line\" dy=\"1.3em\" x=\"0\" y=\"383\">2020</tspan></text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(195.70999999999998,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 10</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(272.86,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 12</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(350,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 14</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(427.14,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 16</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(504.29,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 18</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(581.4300000000001,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 20</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,348.66999999999996)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">2000</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,281.78999999999996)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">3000</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,214.9)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">4000</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,148.01)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">5000</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-ccf1f8\"><g class=\"clips\"/></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"350\" y=\"50\" text-anchor=\"middle\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Number of total daily events.</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"350\" y=\"422\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Event Date</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,42,235)\" x=\"42\" y=\"235\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Number of events</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tm_events_per_day = df_ab_testing.groupby('event_date')['event_name'].agg([\n",
    "    ('n_events', 'count')\n",
    "]).reset_index()\n",
    "tm_events_per_day.head()\n",
    "\n",
    "fig = px.line(tm_events_per_day,\n",
    "              x=\"event_date\",\n",
    "              y=\"n_events\",\n",
    "              title='Number of total daily events.',\n",
    "              labels={\n",
    "                  'n_events': 'Number of events',\n",
    "                  'event_date': 'Event Date'\n",
    "              })\n",
    "\n",
    "fig.show('svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">*Number of total daily events per test groups:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_date</th>\n",
       "      <th>group</th>\n",
       "      <th>n_events</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>A</td>\n",
       "      <td>955</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>B</td>\n",
       "      <td>928</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020-12-08</td>\n",
       "      <td>A</td>\n",
       "      <td>1070</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-12-08</td>\n",
       "      <td>B</td>\n",
       "      <td>1098</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-12-09</td>\n",
       "      <td>A</td>\n",
       "      <td>1084</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  event_date group  n_events\n",
       "0 2020-12-07     A       955\n",
       "1 2020-12-07     B       928\n",
       "2 2020-12-08     A      1070\n",
       "3 2020-12-08     B      1098\n",
       "4 2020-12-09     A      1084"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"450\" style=\"\" viewBox=\"0 0 700 450\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"450\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-9fd504\"><g class=\"clips\"><clipPath id=\"clip9fd504xyplot\" class=\"plotclip\"><rect width=\"518\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clip9fd504x\"><rect x=\"80\" y=\"0\" width=\"518\" height=\"450\"/></clipPath><clipPath class=\"axesclip\" id=\"clip9fd504y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clip9fd504xy\"><rect x=\"80\" y=\"100\" width=\"518\" height=\"270\"/></clipPath></g><g class=\"gradients\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(117,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(191,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(265,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(339,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(413,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(487,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(561,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,346.99)\" d=\"M80,0h518\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,280.96000000000004)\" d=\"M80,0h518\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,214.93)\" d=\"M80,0h518\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,148.89)\" d=\"M80,0h518\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"/><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80, 100)\" clip-path=\"url('#clip9fd504xyplot')\"><g class=\"scatterlayer mlayer\"><g class=\"trace scatter trace6546df\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,252.93L37,237.75L74,235.9L111,220.05L148,216.48L185,147.55L222,121.79L259,70.02L296,108.06L333,122.72L370,127.21L407,117.17L444,112.42L481,78.21L518,16.14\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter tracec6fd94\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,256.5L37,234.05L74,237.48L111,221.24L148,216.09L185,166.83L222,129.45L259,98.02L296,118.49L333,151.64L370,141.87L407,135.92L444,122.98L481,95.91L518,13.5\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(255, 127, 14); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(117,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\"><tspan class=\"line\" dy=\"0em\" x=\"0\" y=\"383\">Dec 8</tspan><tspan class=\"line\" dy=\"1.3em\" x=\"0\" y=\"383\">2020</tspan></text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(191,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 10</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(265,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 12</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(339,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 14</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(413,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 16</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(487,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 18</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(561,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 20</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,346.99)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">1000</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,280.96000000000004)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">1500</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,214.93)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">2000</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,148.89)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">2500</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-9fd504\"><g class=\"clips\"/><clipPath id=\"legend9fd504\"><rect width=\"80\" height=\"64\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(608.36, 100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"80\" height=\"64\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"translate(0, 0)\" clip-path=\"url('#legend9fd504')\"><text class=\"legendtitletext user-select-none\" text-anchor=\"start\" x=\"2\" y=\"15.600000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Test groups:</text><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 30.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">A</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"49.625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 49.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">B</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(255, 127, 14); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"49.625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"350\" y=\"50\" text-anchor=\"middle\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Number of total daily events per group.</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"339\" y=\"422\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Event Date</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,42,235)\" x=\"42\" y=\"235\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Number of events</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tm_events_per_group = df_ab_testing.groupby(['event_date', 'group'])['event_dt'].agg([\n",
    "    ('n_events', 'count')\n",
    "]).reset_index()\n",
    "tm_events_per_group.head()\n",
    "\n",
    "fig = px.line(tm_events_per_group,\n",
    "              x=\"event_date\",\n",
    "              y=\"n_events\",\n",
    "              color='group',\n",
    "              title='Number of total daily events per group.',\n",
    "              labels={\n",
    "                  'n_events': 'Number of events',\n",
    "                  'event_date': 'Event Date',\n",
    "                  'group': 'Test groups:'\n",
    "              })\n",
    "\n",
    "fig.show('svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">*Number of total daily events by event name:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_date</th>\n",
       "      <th>event_name</th>\n",
       "      <th>n_events</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>product_cart</td>\n",
       "      <td>250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>product_page</td>\n",
       "      <td>535</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>purchase</td>\n",
       "      <td>281</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>login</td>\n",
       "      <td>817</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-12-08</td>\n",
       "      <td>product_cart</td>\n",
       "      <td>295</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  event_date    event_name  n_events\n",
       "0 2020-12-07  product_cart       250\n",
       "1 2020-12-07  product_page       535\n",
       "2 2020-12-07      purchase       281\n",
       "3 2020-12-07         login       817\n",
       "4 2020-12-08  product_cart       295"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"450\" style=\"\" viewBox=\"0 0 700 450\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"450\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-212f5c\"><g class=\"clips\"><clipPath id=\"clip212f5cxyplot\" class=\"plotclip\"><rect width=\"471\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clip212f5cx\"><rect x=\"80\" y=\"0\" width=\"471\" height=\"450\"/></clipPath><clipPath class=\"axesclip\" id=\"clip212f5cy\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clip212f5cxy\"><rect x=\"80\" y=\"100\" width=\"471\" height=\"270\"/></clipPath></g><g class=\"gradients\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(147.29000000000002,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(248.21,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(349.14,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(450.07,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,327.39)\" d=\"M80,0h471\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,269.16999999999996)\" d=\"M80,0h471\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,210.95999999999998)\" d=\"M80,0h471\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,152.74)\" d=\"M80,0h471\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"/><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80, 100)\" clip-path=\"url('#clip212f5cxyplot')\"><g class=\"scatterlayer mlayer\"><g class=\"trace scatter trace951f4f\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,256.5L33.64,251.26L67.29,248.23L100.93,244.97L134.57,244.86L168.21,225.76L201.86,220.41L235.5,212.72L269.14,218.78L302.79,224.95L336.43,225.53L403.71,217.61L437.36,212.14L471,192.34\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter tracef17bd6\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,223.32L33.64,213.07L67.29,215.4L100.93,205.5L134.57,202.71L168.21,175.93L201.86,159.74L235.5,141L269.14,152.52L302.79,166.73L336.43,163.12L370.07,161.84L403.71,155.09L437.36,142.74L471,103.5\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(255, 127, 14); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter trace568bf0\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,252.89L33.64,247.88L67.29,248.7L100.93,243.23L134.57,243.46L168.21,229.14L201.86,219.01L235.5,206.55L269.14,219.01L302.79,220.17L336.43,222.85L370.07,222.04L403.71,219.47L437.36,207.36L471,190.83\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(44, 160, 44); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter tracedefcb5\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,190.48L33.64,177.79L67.29,179.07L100.93,169.41L134.57,164.4L168.21,120.39L201.86,96.4L235.5,61.94L269.14,83.48L302.79,104.09L336.43,99.78L370.07,91.74L403.71,89.42L437.36,65.31L471,13.5\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(214, 39, 40); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" style=\"fill: none;\" d=\"M0,0\"/><path class=\"ylines-above crisp\" style=\"fill: none;\" d=\"M0,0\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(147.29000000000002,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\"><tspan class=\"line\" dy=\"0em\" x=\"0\" y=\"383\">Dec 9</tspan><tspan class=\"line\" dy=\"1.3em\" x=\"0\" y=\"383\">2020</tspan></text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(248.21,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 12</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(349.14,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 15</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(450.07,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 18</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(551,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 21</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,327.39)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">500</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,269.16999999999996)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">1000</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,210.95999999999998)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">1500</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,152.74)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">2000</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-212f5c\"><g class=\"clips\"/><clipPath id=\"legend212f5c\"><rect width=\"128\" height=\"102\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(560.4200000000001, 100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\" width=\"128\" height=\"102\" x=\"0\" y=\"0\"/><g class=\"scrollbox\" transform=\"translate(0, 0)\" clip-path=\"url('#legend212f5c')\"><text class=\"legendtitletext user-select-none\" text-anchor=\"start\" x=\"2\" y=\"15.600000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Events:</text><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 30.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">product_cart</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"122.921875\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 49.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">product_page</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(255, 127, 14); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"122.921875\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 68.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">purchase</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(44, 160, 44); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"122.921875\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 87.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">login</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(214, 39, 40); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"122.921875\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\" x=\"0\" y=\"0\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"350\" y=\"50\" text-anchor=\"middle\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Number of total daily events by event name.</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"315.5\" y=\"422\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Event Date</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,42,235)\" x=\"42\" y=\"235\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Number of events</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tm_events_by_name = df_ab_testing.groupby(['event_date', 'event_name'])['event_dt'].agg([\n",
    "    ('n_events', 'count')\n",
    "]).reset_index()\n",
    "tm_events_by_name.head()\n",
    "\n",
    "fig = px.line(tm_events_by_name,\n",
    "              x=\"event_date\",\n",
    "              y=\"n_events\",\n",
    "              color='event_name',\n",
    "              title='Number of total daily events by event name.',\n",
    "              labels={\n",
    "                  'n_events': 'Number of events',\n",
    "                  'event_date': 'Event Date',\n",
    "                  'event_name': 'Events:'\n",
    "              })\n",
    "\n",
    "fig.show('svg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_date</th>\n",
       "      <th>device</th>\n",
       "      <th>n_events</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>Android</td>\n",
       "      <td>808</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>Mac</td>\n",
       "      <td>237</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>PC</td>\n",
       "      <td>480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-12-07</td>\n",
       "      <td>iPhone</td>\n",
       "      <td>358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-12-08</td>\n",
       "      <td>Android</td>\n",
       "      <td>979</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  event_date   device  n_events\n",
       "0 2020-12-07  Android       808\n",
       "1 2020-12-07      Mac       237\n",
       "2 2020-12-07       PC       480\n",
       "3 2020-12-07   iPhone       358\n",
       "4 2020-12-08  Android       979"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"450\" style=\"\" viewBox=\"0 0 700 450\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"450\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-0a332c\"><g class=\"clips\"><clipPath id=\"clip0a332cxyplot\" class=\"plotclip\"><rect width=\"505\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clip0a332cx\"><rect x=\"80\" y=\"0\" width=\"505\" height=\"450\"/></clipPath><clipPath class=\"axesclip\" id=\"clip0a332cy\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"270\"/></clipPath><clipPath class=\"axesclip\" id=\"clip0a332cxy\"><rect x=\"80\" y=\"100\" width=\"505\" height=\"270\"/></clipPath></g><g class=\"gradients\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(116.07,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(188.20999999999998,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(260.36,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(332.5,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(404.64,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(476.79,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(548.9300000000001,0)\" d=\"M0,100v270\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,327.15999999999997)\" d=\"M80,0h505\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,271.81)\" d=\"M80,0h505\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,216.45999999999998)\" d=\"M80,0h505\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,161.1)\" d=\"M80,0h505\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,105.75)\" d=\"M80,0h505\" style=\"stroke: rgb(238, 238, 238); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"/><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80, 100)\" clip-path=\"url('#clip0a332cxyplot')\"><g class=\"scatterlayer mlayer\"><g class=\"trace scatter trace4d617a\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,193.07L36.07,174.13L72.14,175.35L108.21,155.43L144.29,157.75L180.36,112.36L216.43,92.43L252.5,69.74L288.57,83.8L324.64,94.87L360.71,100.29L396.79,93.76L432.86,83.91L468.93,64.54L505,13.5\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter tracefe7975\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,256.28L72.14,253.84L108.21,256.5L144.29,253.62L180.36,244.54L216.43,238.57L252.5,227.05L288.57,237.02L324.64,240.67L360.71,234.91L396.79,239.23L432.86,240.56L468.93,235.02L505,215.32\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(255, 127, 14); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter tracec024f5\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,229.38L36.07,223.62L72.14,224.62L108.21,217.53L144.29,217.42L180.36,190.85L216.43,175.02L252.5,162.18L288.57,171.59L324.64,184.54L360.71,182.88L396.79,177.46L432.86,167.82L468.93,157.64L505,117.12\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(44, 160, 44); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter trace837ead\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,242.88L36.07,236.91L72.14,237.57L108.21,235.02L144.29,228.38L180.36,210.34L216.43,199.15L252.5,176.46L288.57,192.07L324.64,204.47L360.71,202.03L396.79,196.28L432.86,199.6L468.93,183.32L505,173.47\" style=\"vector-effect: non-scaling-stroke; fill: none; stroke: rgb(214, 39, 40); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" style=\"fill: none;\" d=\"M0,0\"/><path class=\"ylines-above crisp\" style=\"fill: none;\" d=\"M0,0\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(116.07,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\"><tspan class=\"line\" dy=\"0em\" x=\"0\" y=\"383\">Dec 8</tspan><tspan class=\"line\" dy=\"1.3em\" x=\"0\" y=\"383\">2020</tspan></text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(188.20999999999998,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 10</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(260.36,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 12</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(332.5,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 14</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(404.64,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 16</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(476.79,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 18</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"383\" transform=\"translate(548.9300000000001,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Dec 20</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,327.15999999999997)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">500</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,271.81)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">1000</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,216.45999999999998)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">1500</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,161.1)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">2000</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,105.75)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">2500</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-0a332c\"><g class=\"clips\"/><clipPath id=\"legend0a332c\"><rect width=\"93\" height=\"102\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(595.1, 100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\" width=\"93\" height=\"102\" x=\"0\" y=\"0\"/><g class=\"scrollbox\" transform=\"translate(0, 0)\" clip-path=\"url('#legend0a332c')\"><text class=\"legendtitletext user-select-none\" text-anchor=\"start\" x=\"2\" y=\"15.600000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Devices:</text><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 30.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Android</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"87.265625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 49.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">Mac</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(255, 127, 14); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"87.265625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 68.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">PC</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(44, 160, 44); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"87.265625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"translate(0, 0)\"><g class=\"traces\" transform=\"translate(0, 87.1)\" style=\"opacity: 1;\"><text class=\"legendtext user-select-none\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">iPhone</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(214, 39, 40); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" pointer-events=\"all\" x=\"0\" y=\"-9.5\" width=\"87.265625\" height=\"19\" style=\"cursor: pointer; fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"350\" y=\"50\" text-anchor=\"middle\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Number of total daily events per device.</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"332.5\" y=\"422\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Event Date</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,42,235)\" x=\"42\" y=\"235\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(68, 68, 68); opacity: 1; font-weight: normal; white-space: pre;\">Number of events</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tm_events_by_device = df_ab_testing.groupby(['event_date', 'device'])['event_dt'].agg([\n",
    "    ('n_events', 'count')\n",
    "]).reset_index()\n",
    "tm_events_by_device.head()\n",
    "\n",
    "fig = px.line(tm_events_by_device,\n",
    "              x=\"event_date\",\n",
    "              y=\"n_events\",\n",
    "              color='device',\n",
    "              title='Number of total daily events per device.',\n",
    "              labels={\n",
    "                  'n_events': 'Number of events',\n",
    "                  'event_date': 'Event Date',\n",
    "                  'device': 'Devices:'\n",
    "              })\n",
    "\n",
    "fig.show('svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">* The total number of events during the test increased significantly, 3 times.\n",
    "* This feature is typical for both Groups A and B, for each event name and for number of events per device.\n",
    "* Most of the events were performed by Android users, with the smallest user group being Mac fans.\n",
    "* It is clear from the graphs that we can say that there were spikes of events on a certain weekday. In our case, it was on Thursdays.\n",
    "* To funnel stages, `product_cart` and `purchase` have almost the same number of events during the test weeks. **I can assume that the abandoned cart percentage is very low, or the 'Buy Now' button works better than the whole funnel.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Think of the possible details in the data that you have to take into account before starting the A/B test?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">* The datasets had users who were assigned to multiple groups and who performed both tests at the same time. We have processed our data to avoid exposure to these factors.\n",
    "* We also checked if the users are equally split.\n",
    "* But there is an external factor that influences the test result. I consider that we have to take into account that the A / B test was carried out on the eve of Christmas. \n",
    "<br>And even if the specific testing time did not last at the same time as the marketing activities, we cannot trust the results 100% due to the well-known increase in consumer activity before the Christmas holidays."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion:\n",
    "\n",
    ">* The test group B did not achieve expected result of increasing total conversion at least 10% on each stage of the funnel `product_page → product_cart → purchase`.\n",
    "* The test group B showed better result on the `purchase` stage.\n",
    "* There is no users who enter both samples.\n",
    "* The number of events per user distributed equally in the samples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate the A/B test results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What can you tell about the A/A test results?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CategoricalDtype(categories=['product_cart', 'product_page', 'purchase', 'login'], ordered=False)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "CategoricalDtype(categories=['A', 'B'], ordered=False)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_ab_testing.event_name.dtypes\n",
    "df_ab_testing.group.dtypes\n",
    "\n",
    "# changing dtype, to avoid issue with CategoricalIndex columns in pandas\n",
    "# https://github.com/pandas-dev/pandas/issues/19136\n",
    "\n",
    "df_ab_testing.event_name = dtype_converter(df_ab_testing.event_name, 'str')\n",
    "df_ab_testing.group = dtype_converter(df_ab_testing.group, 'str')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>group</th>\n",
       "      <th>event_name</th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>diff_n_users</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>login</td>\n",
       "      <td>4520</td>\n",
       "      <td>4438</td>\n",
       "      <td>82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>product_cart</td>\n",
       "      <td>1479</td>\n",
       "      <td>1516</td>\n",
       "      <td>-37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>product_page</td>\n",
       "      <td>3030</td>\n",
       "      <td>2913</td>\n",
       "      <td>117</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>purchase</td>\n",
       "      <td>1612</td>\n",
       "      <td>1479</td>\n",
       "      <td>133</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "group    event_name     A     B  diff_n_users\n",
       "0             login  4520  4438            82\n",
       "1      product_cart  1479  1516           -37\n",
       "2      product_page  3030  2913           117\n",
       "3          purchase  1612  1479           133"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "users_pivot = df_ab_testing.pivot_table(index='event_name',\n",
    "                         values='user_id',\n",
    "                         columns='group',\n",
    "                         aggfunc=lambda x: x.nunique()).reset_index()\n",
    "\n",
    "users_pivot= users_pivot.assign(diff_n_users=lambda x: x.A - x.B)\n",
    "users_pivot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>group</th>\n",
       "      <th>event_name</th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>Total</th>\n",
       "      <th>diff_share_stage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>login</td>\n",
       "      <td>0.215371</td>\n",
       "      <td>0.211464</td>\n",
       "      <td>0.426836</td>\n",
       "      <td>0.39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>product_cart</td>\n",
       "      <td>0.070472</td>\n",
       "      <td>0.072235</td>\n",
       "      <td>0.142707</td>\n",
       "      <td>-0.18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>product_page</td>\n",
       "      <td>0.144375</td>\n",
       "      <td>0.138800</td>\n",
       "      <td>0.283175</td>\n",
       "      <td>0.56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>purchase</td>\n",
       "      <td>0.076809</td>\n",
       "      <td>0.070472</td>\n",
       "      <td>0.147282</td>\n",
       "      <td>0.63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Total</td>\n",
       "      <td>0.504632</td>\n",
       "      <td>0.495368</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.93</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "group    event_name         A         B     Total  diff_share_stage\n",
       "0             login  0.215371  0.211464  0.426836              0.39\n",
       "1      product_cart  0.070472  0.072235  0.142707             -0.18\n",
       "2      product_page  0.144375  0.138800  0.283175              0.56\n",
       "3          purchase  0.076809  0.070472  0.147282              0.63\n",
       "4             Total  0.504632  0.495368  1.000000              0.93"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "users_crosstab = pd.crosstab(index=df_ab_testing.event_name,\n",
    "                             columns=df_ab_testing.group,\n",
    "                             values=df_ab_testing.user_id,\n",
    "                             aggfunc=lambda x: x.nunique(),\n",
    "                             normalize=True, # Normalize by dividing all values by the sum of values.\n",
    "                             margins=True,\n",
    "                             margins_name='Total').reset_index()\n",
    "\n",
    "users_crosstab = users_crosstab.assign(diff_share_stage=lambda x: round((x.A - x.B )* 100, 2))\n",
    "users_crosstab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We can argue that all the criteria we have for a successful A/A test:\n",
    "* As can be seen from the aggregated tables, there is no significant difference between the samples. On Each stage the difference is less then 1%.\n",
    "* For all groups, data on the same event is recorded.\n",
    "* Users remain within their groups until the end of the test. \n",
    "* Users who may see different versions of the page during the test were removed from the groups."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use the z-criterion to check the statistical difference between the proportions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***H0: there is not a statistically significant difference in conversion between the samples A and B***\n",
    "\n",
    "***H1: there is a statistically significant difference in conversion between the samples A and B***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "p-value:  0.321766940707354\n",
      "Fail to Reject H0 for login and groups A B\n",
      "p-value:  0.14702821950355438\n",
      "Fail to Reject H0 for product_cart and groups A B\n",
      "p-value:  0.16608962447812559\n",
      "Fail to Reject H0 for product_page and groups A B\n",
      "p-value:  0.02036540675054077\n",
      "Reject H0 for purchase and groups A B\n"
     ]
    }
   ],
   "source": [
    "for i in users_pivot.event_name.unique():\n",
    "    check_hypothesis(\"A\", \"B\", i, alpha=0.05)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    ">* As a result of checking the hypothesis, we cannot reject H0 for all the events. \n",
    "* Thus, we can accept the equality of the proportions of the population of `A` and `B` groups. It means that the groups were split properly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# General conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ***Datasets:***\n",
    "\n",
    "* The logs were collected in several tables for the period from December 7 to December 30, 2020.\n",
    "* The two types of tests were run simultaneously and overlapped using the same test participants.\n",
    "\n",
    "> ***New interface for EU users test:***\n",
    "\n",
    "* The test was carried out within 14 days, from December 7 to December 21, 2020.\n",
    "* After preprocessing, we got a dataset that contains 51848 different events performed by 8959 unique users.\n",
    "* Users are split equally into two groups:\n",
    "        - `A` control group - 4521 users\n",
    "        - `B` test group - 4438 users\n",
    "* Control group `A` gets the old site version and one test group `B` gets the new ones:        \n",
    "\n",
    "> ***The funnel:***\n",
    "\n",
    "* Consists of 4 events: `login -> product_page -> product_cart -> purchase`\n",
    "* The number of events per user distributed equally in the samples.\n",
    "\n",
    "> ***Performing tests:***\n",
    "\n",
    "* The A/A test shows that the control groups were split properly.\n",
    "* A / B tests show that we cannot reject null hypotheses. In other words, there are no statistically significant differences in conversions between two groups across all 4 events.\n",
    "\n",
    "> ***External factors:***\n",
    "\n",
    "* The A / B test was carried out on the eve of Christmas during the period of increased buying activity before the Christmas holidays. Thus, we cannot trust the results 100%.\n",
    "\n",
    "\n",
    "> ***Test result:***\n",
    "\n",
    "* The test group B did not achieve the expected result of increasing total conversion at least 10% on each stage of the funnel `product_page → product_cart → purchase`.\n",
    "* Moreover, total conversion in group B is almost the same or less than total conversion in group A on stages product_page -> product_cart.\n",
    "* Only at the last stage,`purchase`, test group B showed a 1.11% better result than in group A and than at the `product_cart` stage in their group.\n",
    "<br>It means that some EU users made purchasing in one-click using the 'Buy Now' button.\n",
    "\n",
    "\n",
    "> ***Recommendation:***\n",
    "\n",
    "* It can be assumed that **EU users prefer to shop in the shortest and fastest way**, skipping stages in the usual sales funnel order. This hypothesis should be tested."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Thank you in advance for the reviewing*** 😊"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "303.2px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}