NB1-PoolExploration-0x8b6-V1.0.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NB #1:  Pool Exploration \n",
    "\n",
    "### Pool address: 0x8b6e6E7B5b3801FEd2CaFD4b22b8A16c2F2Db21a  -- 80% WETH - 20% DAI \n",
    "\n",
    "The Balancer Pool cadCAD model provides a core infrastructure for simulating Balancer Pools in the **Token Engineering Design and Verification process**.\n",
    "\n",
    "It allows to simulate Balancer Pool transactions and verify new designs and application cases.  \n",
    "In this notebook we plug **historical on-chain data** to a cadCAD model, and analyse **Pool Power** and **Pool Characteristics**.\n",
    "\n",
    "For more information check out the [Balancer Simulations documentation](https://token-engineering-balancer.gitbook.io/balancer-simulations/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A. System Context"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A1.1 System Specification\n",
    "System specification details are available in the Balancer Simulations documentation:\n",
    "- [Differential Specification](https://token-engineering-balancer.gitbook.io/balancer-simulations/balancer-simulations/v10nboverview)\n",
    "- [Mathematical Specification](https://token-engineering-balancer.gitbook.io/balancer-simulations/additional-code-and-instructions/balancer-the-python-edition/balancer_math.py)\n",
    "- [Model Architecture](https://token-engineering-balancer.gitbook.io/balancer-simulations/balancer-simulations/v10nboverview)\n",
    "- [Naming Convention](https://token-engineering-balancer.gitbook.io/balancer-simulations/additional-code-and-instructions/naming-convention)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# B. cadCAD Simulations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.1 Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd \n",
    "from cadCAD.configuration.utils import config_sim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.2 Initialize Pool\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section you specify the Balancer Pool to be analyzed and run scripts to produce the'genesis_state', based on on-chain data. \n",
    "\n",
    "More information here: [Balancer Simulations documentation](https://token-engineering-balancer.gitbook.io/balancer-simulations/additional-code-and-instructions/onchaintransactions).  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pprint\n",
    "pp = pprint.PrettyPrinter(indent=4)\n",
    "\n",
    "from decimal import Decimal\n",
    "\n",
    "# Spot price reference must be a symbol of a token in the pool in ALL_CAPS, you can ignore the spot price parameter for the simulations in this notebook.\n",
    "parameters = {\n",
    "    'spot_price_reference': ['DAI'],\n",
    "    'decoding_type': ['CONTRACT_CALL']\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import Pool Transactions and Initialize Pool state\n",
    "\n",
    "Genesis state, pool transactions, and (external) USD token prices are pulled from on-chain/API data collected in a .json file.  \n",
    "**Choose *your* Balancer Pool, and produce this .json file first, then add the file path below.**\n",
    "\n",
    "Learn more how to create this file using parsing scripts in the [documentation](https://token-engineering-balancer.gitbook.io/balancer-simulations/v/master/additional-code-and-instructions/onchaintransactions).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from model.genesis_states import generate_initial_state\n",
    "\n",
    "initial_values = generate_initial_state(initial_values_json='data/0x8b6e6e7b5b3801fed2cafd4b22b8a16c2f2db21a-initial_pool_states-prices.json', spot_price_base_currency=parameters['spot_price_reference'][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### State Variables and Initial Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('## State Variables')\n",
    "print('# Pool')\n",
    "pool = initial_values['pool']\n",
    "pp.pprint(initial_values)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### External USD Token Prices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('# External token prices, initial state')\n",
    "token_prices = initial_values['token_prices']\n",
    "pp.pprint(token_prices)\n",
    "\n",
    "print('# Action Type')\n",
    "action_type = initial_values['action_type']\n",
    "pp.pprint(action_type)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.3 State Update Functions & Policies\n",
    "\n",
    "Balancer Simulations replicate Balancer Pool Transactions in state update functions.  \n",
    "For a detailed description, please visit the [Balancer Simulations documentation](https://token-engineering-balancer.gitbook.io/balancer-simulations/additional-code-and-instructions/balancer-the-python-edition)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.4 Partial State Update Blocks\n",
    "\n",
    "Partial State Update Blocks combine the following steps:  \n",
    "1. **Initialize pool**, generate genesis state\n",
    "2. Compute subsequent **actions in discrete timesteps**, store datetime and update the pool state variables\n",
    "3. Update external **USD prices in discrete timesteps**, and store datetime  \n",
    "\n",
    "For more information please visit the [Balancer Simulations documentation/Model Overview](https://token-engineering-balancer.gitbook.io/balancer-simulations/v/master/balancer-simulations/v10nboverview).  \n",
    "\n",
    "To inject historical on-chain transactions to the model, reference the **actions.json of your pool below**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from model.partial_state_update_block import generate_partial_state_update_blocks\n",
    "\n",
    "result = generate_partial_state_update_blocks('data/0x8b6e6e7b5b3801fed2cafd4b22b8a16c2f2db21a-actions-prices.json')\n",
    "partial_state_update_blocks = result['partial_state_update_blocks']\n",
    "pp.pprint(partial_state_update_blocks)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.5 Configuration\n",
    "\n",
    "[cadCAD simulations](https://github.com/cadCAD-org/cadCAD/blob/master/documentation/README.md) support Monte Carlo runs, and parameter sweeps which are not applied in this notebook and can be ignored here.  \n",
    "Timesteps 'T' are defined by your pool's .json file, no need to make edits here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "steps_number = result['steps_number']\n",
    "print('# Steps ', steps_number)\n",
    "sim_config = config_sim(\n",
    "    {\n",
    "        'N': 1,  # number of monte carlo runs\n",
    "        'T': range(steps_number - 1),  # number of timesteps\n",
    "        'M': parameters,  # simulation parameters\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.6 Execution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from model.sim_runner import *\n",
    "\n",
    "df = run(initial_values, partial_state_update_blocks, sim_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B1.7 Simulation Output Preparation\n",
    "\n",
    "Post-processing (utils.py) splits up state variable dictionaries, and adds metrics to the data frame, such as \n",
    "- `token_total_value` (Total Value Locked in the pool)\n",
    "- `invariant` (Invariant V of the pool)\n",
    "- `generated_fees_(tokensymbol)` (fee collected per transaction)\n",
    "- `token_k_values`  (token value in USD)  \n",
    "based on  \n",
    "- `token_k_balances`\n",
    "- `token_k_price`  (external price feed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from model.parts.utils import post_processing\n",
    "\n",
    "p_df = post_processing(df, include_spot_prices=False)\n",
    "\n",
    "p_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# C. Simulation Outcome & Pool Exploration\n",
    "\n",
    "The plots below offer some keys metrics for pool analysis.\n",
    "\n",
    "**a) Pool Energy:**  \n",
    "- C1.1 TVL & Invariant V\n",
    "- C1.2 Token Balances (individual token balances, over time)\n",
    "\n",
    "**b) Pool Characteristics:**  \n",
    "- C1.3 Sources of Growth (BPT, Fees)\n",
    "- C1.4 Action Types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import plotly.express as px\n",
    "import plotly.graph_objects as go\n",
    "from plotly.subplots import make_subplots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Observation Time\")\n",
    "print(f\"Start: {p_df['change_datetime'].min()}\")\n",
    "print(f\"End: {p_df['change_datetime'].max()}\")\n",
    "print(f\"Total Observation Period: {p_df['change_datetime'].max() - p_df['change_datetime'].min()}\")\n",
    "print(\"\\n\")\n",
    "print(f\"Total No. of Timesteps (incl. Price Updates): {(p_df.iloc[-1]['timestep'])}\")\n",
    "print(f\"Total No. of Transactions: {len(p_df[p_df.action_type != 'external_price_update'])}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C1.1 TVL & Invariant V"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TVL vs. Invariant V (total observation period)\n",
    "fig = make_subplots(specs=[[{'secondary_y': True}]])\n",
    "fig.add_trace(go.Scatter(x=p_df.timestep,y=p_df.tvl, line=dict(color='#f36315'), name=\"TVL\"), secondary_y=False)\n",
    "fig.add_trace(go.Scatter(x=p_df.timestep, y=p_df.invariant, line=dict(color='#2C1839'), name=\"Invariant V\"), secondary_y=True)\n",
    "fig.update_layout(title_text='<b>TVL vs. Invariant V</b>')\n",
    "fig.update_xaxes(title_text='timestep')\n",
    "fig.update_yaxes(title_text='<b>TVL</b> in USD', secondary_y=False)\n",
    "fig.update_yaxes(title_text='<b>Invariant V', secondary_y=True, range=[183000, 197000])\n",
    "fig.show()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Min/Max overall (total observation period)\n",
    "print(\"TVL min/max\")\n",
    "print(f\"Min: {p_df['tvl'].min()} at timestep {p_df['tvl'].argmin()}\")\n",
    "print(f\"Max: {p_df['tvl'].max()} at timestep {p_df['tvl'].argmax()}\")\n",
    "print(\"\\n\")\n",
    "print(f\"Invariant V min/max\")\n",
    "print(f\"Min: {p_df['invariant'].min()} at timestep {p_df['invariant'].argmin()}\")\n",
    "print(f\"Max: {p_df['invariant'].max()} at timestep {p_df['invariant'].argmax()}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Min/Max before LP exit\n",
    "\n",
    "#define observation period before LP exited\n",
    "rp_df = p_df.loc[0:44000,:]\n",
    "\n",
    "print(\"TVL min/max BEFORE LP EXIT\")\n",
    "print(f\"Min: {rp_df['tvl'].min()} at timestep {rp_df['tvl'].argmin()}\")\n",
    "print(f\"Max: {rp_df['tvl'].max()} at timestep {rp_df['tvl'].argmax()}\")\n",
    "print(\"\\n\")\n",
    "print(f\"Invariant V min/max BEFORE LP EXIT\")\n",
    "print(f\"Min: {rp_df['invariant'].min()} at timestep {rp_df['invariant'].argmin()}\")\n",
    "print(f\"Max: {rp_df['invariant'].max()} at timestep {rp_df['invariant'].argmax()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Observation:**\n",
    "- after timestep 44.000 the pool lost almost 100% of its value, since liquidity providers exited\n",
    "- before LP exited, TVL grew significantly, mainly driven by WETH value growth (and not growth of the pool itself)\n",
    "- during this period, the actual \"pool energy\" represented by the Invariant V increased only moderately"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C1.2 Token Balances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#balance growth % (total observation period)\n",
    "dai_growth = ((p_df.iloc[-1][['token_dai_balance']])-(p_df.iloc[0][['token_dai_balance']]))*100/(p_df.iloc[0][['token_dai_balance']])\n",
    "weth_growth = ((p_df.iloc[-1][['token_weth_balance']])-(p_df.iloc[0][['token_weth_balance']]))*100/(p_df.iloc[0][['token_weth_balance']])\n",
    "growth = list(zip(dai_growth, weth_growth)) \n",
    "g_df = pd.DataFrame(growth, columns = ['dai_growth','weth_growth' ]).transpose(copy=True).reset_index()\n",
    "g_df.columns =['token', 'growth']\n",
    "print(g_df)\n",
    "#plot\n",
    "fig = make_subplots(rows=2, cols=2, subplot_titles=('DAI Growth', 'Balance Growth in %', 'WETH Growth'))\n",
    "fig.add_trace(go.Scatter(x=p_df.timestep, y=p_df.token_dai_balance, line=dict(color='#4675ed'), name='token_dai_balance'), row=1, col=1)\n",
    "fig.add_trace(go.Scatter(x=p_df.timestep, y=p_df.token_weth_balance, line=dict(color='#4145ab'), name='token_weth_balance'), row=2, col=1)\n",
    "fig.add_trace(go.Bar(y=g_df.growth, x=g_df.token, marker_color=['#4675ed','#4145ab']), row=1, col=2)\n",
    "fig.update_layout(height=600, width=1000, showlegend=False, title_text='<b>Balance Growth</b> in observation period')\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C1.3 Sources of Growth "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Pool Shares (total observation period)\n",
    "fig = px.line(p_df, x=p_df.timestep,y=p_df.pool_shares)\n",
    "fig.update_layout(height=300, width=1000, title_text='<b>Pool Shares (BPT)</b>')\n",
    "fig.update_xaxes(title_text='timestep')\n",
    "fig.update_yaxes(title_text='<b>pool shares</b> in #', range=[-0.05,102.00])\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Fees (total observation period)\n",
    "fig = make_subplots(rows=2, cols=1)\n",
    "fig.add_trace(go.Scatter(x=p_df.timestep, y=p_df.generated_fees_dai, line=dict(color='#4675ed'), name='fees DAI'), row=1, col=1)\n",
    "fig.add_trace(go.Scatter(x=p_df.timestep, y=p_df.generated_fees_weth, line=dict(color='#4145ab'), name='fees WETH'), row=2, col=1)\n",
    "fig.update_yaxes(title_text='token amount', secondary_y=False)\n",
    "fig.update_layout(height=500, width=1000, title_text='<b>Fees Generated</b> per transaction')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DAI fee events vs. ETH fee events (total observation period)\n",
    "WETH_fee_events = len(p_df[p_df.generated_fees_weth > 0])\n",
    "DAI_fee_events = len(p_df[p_df.generated_fees_dai > 0])\n",
    "values = [DAI_fee_events, WETH_fee_events]\n",
    "labels = ['DAI fees', 'WETH fees']\n",
    "fig = go.Figure(data=[go.Pie(labels=labels, values=values)])\n",
    "fig.update_traces(hoverinfo='label+percent', marker=dict(colors=['#4675ed','#4145ab']))\n",
    "fig.update_layout(height=500, width=1000, title_text='<b>Fee generating events</b> in total observation time')\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#proportions of fee value contribution, in fixed USD value (total observation period)\n",
    "daif = p_df.groupby('action_type').sum()['generated_fees_dai'].drop(['exit', 'join', 'external_price_update','pool_creation'])\n",
    "wethf = p_df.groupby('action_type').sum()['generated_fees_weth'].drop(['exit', 'join', 'external_price_update','pool_creation'])\n",
    "fee_v = pd.concat([daif, wethf], axis=1)\n",
    "\n",
    "#calculate USD value (define USD value, in this case locked to initial state observation time)\n",
    "fee_v['DAI_fee_value'] = fee_v.generated_fees_dai*1.0053414509361551 #define DAI price\n",
    "fee_v['WETH_fee_value'] = fee_v.generated_fees_weth*594.3526451552318 #define WETH price\n",
    "\n",
    "#add total \n",
    "fee_v = fee_v.append(fee_v.sum().rename('total'))\n",
    "print(fee_v)\n",
    "\n",
    "#plot pie chart comparing ETH fees vs DAI fees\n",
    "values = [fee_v.loc['total']['DAI_fee_value'], fee_v.loc['total']['WETH_fee_value']] \n",
    "labels = ['DAI_fees', 'WETH_fees']\n",
    "fig = go.Figure(data=[go.Pie(labels=labels, values=values)])\n",
    "fig.update_traces(hoverinfo='label+percent', marker=dict(colors=['#4675ed','#4145ab']))\n",
    "fig.update_layout(height=500, width=1000, title_text=\"<b>Fee value contribution</b> in USD, in total observation time\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# FEE PER DAY/in fixed USD value (total observation period)\n",
    "\n",
    "#calculate USD value (define USD value, in this case locked to initial state observation time)\n",
    "fee_t = p_df[['change_datetime']].dropna()\n",
    "fee_t['DAI_fee_value'] = p_df.generated_fees_dai*1.0053414509361551 #define DAI price\n",
    "fee_t['WETH_fee_value'] = p_df.generated_fees_weth*594.3526451552318 #define WETH price\n",
    "daily = fee_t.groupby(pd.Grouper(key='change_datetime',freq='D')).sum() #sum per week\n",
    "\n",
    "#plot value per day\n",
    "fig = go.Figure(data=[go.Bar(name='DAI fees', x=daily.index, y=daily.DAI_fee_value, marker_color='#4675ed', offsetgroup=0), go.Bar(name='WETH fees', x=daily.index, y=daily.WETH_fee_value, marker_color='#4145ab', offsetgroup=1)], layout=go.Layout(title='<b>Fees per day</b> in USD', yaxis_title='USD value'))\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C1.4 Action Types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Action Types by proportion (total observation period)\n",
    "actions = p_df.groupby('action_type').size().to_frame('count').reset_index()\n",
    "fig = px.pie(actions, values='count', names='action_type', title='<b>Action Types</b>', hole=.3, color_discrete_sequence=px.colors.sequential.Turbo)\n",
    "fig.show()\n",
    "print(actions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#Action Type per timestep (total observation period)\n",
    "fig = px.scatter(p_df, x='timestep', y='invariant', color='action_type', color_discrete_sequence=px.colors.sequential.Turbo, category_orders={'action_type': ['swap', 'external_price_update', 'join_swap', 'join', 'exit_swap', 'exit', 'pool_creation']})\n",
    "fig.update_layout(height=400, width=1000, title_text=\"<b>Action Type / Timestep</b>\")\n",
    "fig.update_xaxes(rangeslider_visible=True)\n",
    "fig.update_yaxes(range=[183000, 189000])\n",
    "fig.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}