07_get_many_historical_forecasts.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting several years of historical data\n",
    "This notebook goes through the process of retrieving several years of forecast and actual data. \n",
    "\n",
    "### IMPORTANT - READ THIS:\n",
    "**This can take a very long time, up to many days, and processing datasets can be very memory intensive.** \n",
    "\n",
    "Resulting netCDF files take up about 30 GB of disk space, and are not included in this repository. Later notebooks only use the pickled dataframes that are saved from this notebook. Those pickle files (.pkl) _are included in this repository_, so **if you don't want to download all the forecasts and actuals, you can skip this notebook and move on to the next notebook, [08_prep_data_for_machine_learning.ipynb](08_prep_data_for_machine_learning.ipynb)**. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sffp_tools.herbie_gefs_tools import get_many_tcc_fcasts_fastherbie\n",
    "from sffp_tools.herbie_hrrr_tools import get_many_hrrr_fcasts_fastherbie\n",
    "import gc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load plant specifications:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "plants_df = pd.read_csv('pv_fleet_metadata_rev3.csv') # read in as a Pandas DataFrame\n",
    "plants_df = plants_df[plants_df['ac_capacity']>=10].copy().reset_index()\n",
    "num_plants = len(plants_df.index) # number of plants"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get GEFS data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loaded gefs_total_cloud_cover_f27-f45_3h_2020123106-2021010906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021011006-2021011906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021012006-2021012906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021013006-2021020806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021020906-2021021806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021021906-2021022806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021030106-2021031006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021031106-2021032006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021032106-2021033006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021033106-2021040906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021041006-2021041906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021042006-2021042906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021043006-2021050906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021051006-2021051906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021052006-2021052906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021053006-2021060806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021060906-2021061806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021061906-2021062806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021062906-2021070806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021070906-2021071806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021071906-2021072806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021072906-2021080706_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021080806-2021081706_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021081806-2021082706_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021082806-2021090606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021090706-2021091606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021091706-2021092606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021092706-2021100606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021100706-2021101606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021101706-2021102606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021102706-2021110506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021110606-2021111506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021111606-2021112506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021112606-2021120506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021120606-2021121506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021121606-2021122506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2021122606-2022010406_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022010506-2022011406_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022011506-2022012406_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022012506-2022020306_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022020406-2022021306_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022021406-2022022306_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022022406-2022030506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022030606-2022031506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022031606-2022032506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022032606-2022040406_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022040506-2022041406_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022041506-2022042406_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022042506-2022050406_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022050506-2022051406_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022051506-2022052406_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022052506-2022060306_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022060406-2022061306_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022061406-2022062306_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022062406-2022070306_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022070406-2022071306_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022071406-2022072306_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022072406-2022080206_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022080306-2022081206_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022081306-2022082206_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022082306-2022090106_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022090206-2022091106_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022091206-2022092106_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022092206-2022100106_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022100206-2022101106_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022101206-2022102106_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022102206-2022103106_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022110106-2022111006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022111106-2022112006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022112106-2022113006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022120106-2022121006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022121106-2022122006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022122106-2022123006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2022123106-2023010906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023011006-2023011906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023012006-2023012906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023013006-2023020806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023020906-2023021806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023021906-2023022806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023030106-2023031006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023031106-2023032006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023032106-2023033006_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023033106-2023040906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023041006-2023041906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023042006-2023042906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023043006-2023050906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023051006-2023051906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023052006-2023052906_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023053006-2023060806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023060906-2023061806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023061906-2023062806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023062906-2023070806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023070906-2023071806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023071906-2023072806_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023072906-2023080706_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023080806-2023081706_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023081806-2023082706_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023082806-2023090606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023090706-2023091606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023091706-2023092606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023092706-2023100606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023100706-2023101606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023101706-2023102606_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023102706-2023110506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023110606-2023111506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023111606-2023112506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023112606-2023120506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023120606-2023121506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023121606-2023122506_1D_step_30_members.nc\n",
      "loaded gefs_total_cloud_cover_f27-f45_3h_2023122606-2024010406_1D_step_30_members.nc\n"
     ]
    }
   ],
   "source": [
    "gc.collect() \n",
    "\n",
    "query_len_days = 10\n",
    "\n",
    "num_members = 30 # Number of ensemble members to pull, max of 30\n",
    "freq='1D' # how often to pull a new forecast\n",
    "fxx_interval = 3 # time step of the forecast, GEFS has 3 hour intervals\n",
    "fxx_range = range(27,46,fxx_interval)\n",
    "\n",
    "# lat, lon extent for region of interest\n",
    "# (left lon, right lon, lower lat, upper lat)\n",
    "region_extent = [-92, -80, 29, 36]\n",
    "\n",
    "start = '2020-12-31T06:00:00'\n",
    "end = '2024-01-01T06:00'\n",
    "\n",
    "save_dir = 'nc_files/gefs/'\n",
    "remove_grib=True\n",
    "w_avg_tcc = get_many_tcc_fcasts_fastherbie(start, end, freq, query_len_days,\n",
    "                                        fxx_range, plants_df, region_extent,\n",
    "                                        num_members=num_members,\n",
    "                                        save=True, save_dir=save_dir,\n",
    "                                        remove_grib=remove_grib)\n",
    "\n",
    "# clean up indices and localize to UTC\n",
    "w_avg_tcc = w_avg_tcc.reset_index().set_index('valid_time_utc_end_of_interval').tz_localize('UTC')\n",
    "\n",
    "# save as pickle file\n",
    "w_avg_tcc.to_pickle('data_files/w_avg_tcc_2021-2023.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>level_0</th>\n",
       "      <th>time_utc</th>\n",
       "      <th>step</th>\n",
       "      <th>tcc_std</th>\n",
       "      <th>tcc_p01</th>\n",
       "      <th>tcc_p02</th>\n",
       "      <th>tcc_p03</th>\n",
       "      <th>tcc_p04</th>\n",
       "      <th>tcc_p05</th>\n",
       "      <th>tcc_p06</th>\n",
       "      <th>...</th>\n",
       "      <th>tcc_p21</th>\n",
       "      <th>tcc_p22</th>\n",
       "      <th>tcc_p23</th>\n",
       "      <th>tcc_p24</th>\n",
       "      <th>tcc_p25</th>\n",
       "      <th>tcc_p26</th>\n",
       "      <th>tcc_p27</th>\n",
       "      <th>tcc_p28</th>\n",
       "      <th>tcc_p29</th>\n",
       "      <th>tcc_p30</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>valid_time_utc_end_of_interval</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2021-01-01 07:00:00+00:00</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-12-31 06:00:00</td>\n",
       "      <td>1 days 03:00:00</td>\n",
       "      <td>0.657303</td>\n",
       "      <td>98.406453</td>\n",
       "      <td>99.825122</td>\n",
       "      <td>99.723112</td>\n",
       "      <td>99.455641</td>\n",
       "      <td>98.431235</td>\n",
       "      <td>99.197912</td>\n",
       "      <td>...</td>\n",
       "      <td>98.459086</td>\n",
       "      <td>98.447650</td>\n",
       "      <td>98.465006</td>\n",
       "      <td>99.460350</td>\n",
       "      <td>99.642682</td>\n",
       "      <td>98.839168</td>\n",
       "      <td>99.351640</td>\n",
       "      <td>98.563921</td>\n",
       "      <td>99.583995</td>\n",
       "      <td>98.303555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-01-01 08:00:00+00:00</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-12-31 06:00:00</td>\n",
       "      <td>1 days 03:00:00</td>\n",
       "      <td>0.657303</td>\n",
       "      <td>98.406453</td>\n",
       "      <td>99.825122</td>\n",
       "      <td>99.723112</td>\n",
       "      <td>99.455641</td>\n",
       "      <td>98.431235</td>\n",
       "      <td>99.197912</td>\n",
       "      <td>...</td>\n",
       "      <td>98.459086</td>\n",
       "      <td>98.447650</td>\n",
       "      <td>98.465006</td>\n",
       "      <td>99.460350</td>\n",
       "      <td>99.642682</td>\n",
       "      <td>98.839168</td>\n",
       "      <td>99.351640</td>\n",
       "      <td>98.563921</td>\n",
       "      <td>99.583995</td>\n",
       "      <td>98.303555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-01-01 09:00:00+00:00</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-12-31 06:00:00</td>\n",
       "      <td>1 days 03:00:00</td>\n",
       "      <td>0.657303</td>\n",
       "      <td>98.406453</td>\n",
       "      <td>99.825122</td>\n",
       "      <td>99.723112</td>\n",
       "      <td>99.455641</td>\n",
       "      <td>98.431235</td>\n",
       "      <td>99.197912</td>\n",
       "      <td>...</td>\n",
       "      <td>98.459086</td>\n",
       "      <td>98.447650</td>\n",
       "      <td>98.465006</td>\n",
       "      <td>99.460350</td>\n",
       "      <td>99.642682</td>\n",
       "      <td>98.839168</td>\n",
       "      <td>99.351640</td>\n",
       "      <td>98.563921</td>\n",
       "      <td>99.583995</td>\n",
       "      <td>98.303555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-01-01 10:00:00+00:00</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-12-31 06:00:00</td>\n",
       "      <td>1 days 06:00:00</td>\n",
       "      <td>1.346134</td>\n",
       "      <td>97.210989</td>\n",
       "      <td>99.084062</td>\n",
       "      <td>98.274144</td>\n",
       "      <td>98.806474</td>\n",
       "      <td>96.843043</td>\n",
       "      <td>99.043834</td>\n",
       "      <td>...</td>\n",
       "      <td>97.420095</td>\n",
       "      <td>97.805882</td>\n",
       "      <td>98.418992</td>\n",
       "      <td>98.181498</td>\n",
       "      <td>97.591987</td>\n",
       "      <td>98.777332</td>\n",
       "      <td>99.539623</td>\n",
       "      <td>94.137960</td>\n",
       "      <td>99.059683</td>\n",
       "      <td>97.906520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2021-01-01 11:00:00+00:00</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-12-31 06:00:00</td>\n",
       "      <td>1 days 06:00:00</td>\n",
       "      <td>1.346134</td>\n",
       "      <td>97.210989</td>\n",
       "      <td>99.084062</td>\n",
       "      <td>98.274144</td>\n",
       "      <td>98.806474</td>\n",
       "      <td>96.843043</td>\n",
       "      <td>99.043834</td>\n",
       "      <td>...</td>\n",
       "      <td>97.420095</td>\n",
       "      <td>97.805882</td>\n",
       "      <td>98.418992</td>\n",
       "      <td>98.181498</td>\n",
       "      <td>97.591987</td>\n",
       "      <td>98.777332</td>\n",
       "      <td>99.539623</td>\n",
       "      <td>94.137960</td>\n",
       "      <td>99.059683</td>\n",
       "      <td>97.906520</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                level_0            time_utc            step  \\\n",
       "valid_time_utc_end_of_interval                                                \n",
       "2021-01-01 07:00:00+00:00             0 2020-12-31 06:00:00 1 days 03:00:00   \n",
       "2021-01-01 08:00:00+00:00             0 2020-12-31 06:00:00 1 days 03:00:00   \n",
       "2021-01-01 09:00:00+00:00             0 2020-12-31 06:00:00 1 days 03:00:00   \n",
       "2021-01-01 10:00:00+00:00             0 2020-12-31 06:00:00 1 days 06:00:00   \n",
       "2021-01-01 11:00:00+00:00             0 2020-12-31 06:00:00 1 days 06:00:00   \n",
       "\n",
       "                                 tcc_std    tcc_p01    tcc_p02    tcc_p03  \\\n",
       "valid_time_utc_end_of_interval                                              \n",
       "2021-01-01 07:00:00+00:00       0.657303  98.406453  99.825122  99.723112   \n",
       "2021-01-01 08:00:00+00:00       0.657303  98.406453  99.825122  99.723112   \n",
       "2021-01-01 09:00:00+00:00       0.657303  98.406453  99.825122  99.723112   \n",
       "2021-01-01 10:00:00+00:00       1.346134  97.210989  99.084062  98.274144   \n",
       "2021-01-01 11:00:00+00:00       1.346134  97.210989  99.084062  98.274144   \n",
       "\n",
       "                                  tcc_p04    tcc_p05    tcc_p06  ...  \\\n",
       "valid_time_utc_end_of_interval                                   ...   \n",
       "2021-01-01 07:00:00+00:00       99.455641  98.431235  99.197912  ...   \n",
       "2021-01-01 08:00:00+00:00       99.455641  98.431235  99.197912  ...   \n",
       "2021-01-01 09:00:00+00:00       99.455641  98.431235  99.197912  ...   \n",
       "2021-01-01 10:00:00+00:00       98.806474  96.843043  99.043834  ...   \n",
       "2021-01-01 11:00:00+00:00       98.806474  96.843043  99.043834  ...   \n",
       "\n",
       "                                  tcc_p21    tcc_p22    tcc_p23    tcc_p24  \\\n",
       "valid_time_utc_end_of_interval                                               \n",
       "2021-01-01 07:00:00+00:00       98.459086  98.447650  98.465006  99.460350   \n",
       "2021-01-01 08:00:00+00:00       98.459086  98.447650  98.465006  99.460350   \n",
       "2021-01-01 09:00:00+00:00       98.459086  98.447650  98.465006  99.460350   \n",
       "2021-01-01 10:00:00+00:00       97.420095  97.805882  98.418992  98.181498   \n",
       "2021-01-01 11:00:00+00:00       97.420095  97.805882  98.418992  98.181498   \n",
       "\n",
       "                                  tcc_p25    tcc_p26    tcc_p27    tcc_p28  \\\n",
       "valid_time_utc_end_of_interval                                               \n",
       "2021-01-01 07:00:00+00:00       99.642682  98.839168  99.351640  98.563921   \n",
       "2021-01-01 08:00:00+00:00       99.642682  98.839168  99.351640  98.563921   \n",
       "2021-01-01 09:00:00+00:00       99.642682  98.839168  99.351640  98.563921   \n",
       "2021-01-01 10:00:00+00:00       97.591987  98.777332  99.539623  94.137960   \n",
       "2021-01-01 11:00:00+00:00       97.591987  98.777332  99.539623  94.137960   \n",
       "\n",
       "                                  tcc_p29    tcc_p30  \n",
       "valid_time_utc_end_of_interval                        \n",
       "2021-01-01 07:00:00+00:00       99.583995  98.303555  \n",
       "2021-01-01 08:00:00+00:00       99.583995  98.303555  \n",
       "2021-01-01 09:00:00+00:00       99.583995  98.303555  \n",
       "2021-01-01 10:00:00+00:00       99.059683  97.906520  \n",
       "2021-01-01 11:00:00+00:00       99.059683  97.906520  \n",
       "\n",
       "[5 rows x 34 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w_avg_tcc.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get HRRR actuals (F00) for 2021, 2022, and 2023. We will get each year separately due to memory constraints with processing the xarray datasets. Running `gc.collect()` in between each year may help with memory issues. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loaded hrrr_solar_f0-f0_1h_2021010106-2021010705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021010706-2021011305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021011306-2021011905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021011906-2021012505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021012506-2021013105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021013106-2021020605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021020606-2021021205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021021206-2021021805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021021806-2021022405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021022406-2021030205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021030206-2021030805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021030806-2021031405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021031406-2021032005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021032006-2021032605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021032606-2021040105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021040106-2021040705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021040706-2021041305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021041306-2021041905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021041906-2021042505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021042506-2021050105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021050106-2021050705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021050706-2021051305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021051306-2021051905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021051906-2021052505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021052506-2021053105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021053106-2021060605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021060606-2021061205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021061206-2021061805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021061806-2021062405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021062406-2021063005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021063006-2021070605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021070606-2021071205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021071206-2021071805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021071806-2021072405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021072406-2021073005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021073006-2021080505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021080506-2021081105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021081106-2021081705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021081706-2021082305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021082306-2021082905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021082906-2021090405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021090406-2021091005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021091006-2021091605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021091606-2021092205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021092206-2021092805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021092806-2021100405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021100406-2021101005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021101006-2021101605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021101606-2021102205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021102206-2021102805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021102806-2021110305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021110306-2021110905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021110906-2021111505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021111506-2021112105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021112106-2021112705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021112706-2021120305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021120306-2021120905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021120906-2021121505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021121506-2021122105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021122106-2021122705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2021122706-2022010205_1h_step.nc\n"
     ]
    }
   ],
   "source": [
    "gc.collect()\n",
    "\n",
    "start = '2021-01-01 06:00'\n",
    "end = '2022-01-02 05:00'\n",
    "\n",
    "freq = '1h'\n",
    "fxx_range = range(0,1)\n",
    "query_len_days = 6\n",
    "save_dir='nc_files/hrrr/'\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs')\n",
    "\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled')\n",
    "p_ac_cs_all = p_ac_cs_all.reset_index().set_index('time_center_labeled')\n",
    "\n",
    "ds.to_netcdf('nc_files/ds_act_2021.nc')\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_act_2021.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loaded hrrr_solar_f0-f0_1h_2022010206-2022010805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022010806-2022011405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022011406-2022012005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022012006-2022012605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022012606-2022020105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022020106-2022020705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022020706-2022021305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022021306-2022021905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022021906-2022022505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022022506-2022030305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022030306-2022030905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022030906-2022031505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022031506-2022032105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022032106-2022032705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022032706-2022040205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022040206-2022040805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022040806-2022041405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022041406-2022042005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022042006-2022042605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022042606-2022050205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022050206-2022050805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022050806-2022051405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022051406-2022052005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022052006-2022052605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022052606-2022060105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022060106-2022060705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022060706-2022061305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022061306-2022061905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022061906-2022062505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022062506-2022070105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022070106-2022070705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022070706-2022071305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022071306-2022071905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022071906-2022072505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022072506-2022073105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022073106-2022080605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022080606-2022081205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022081206-2022081805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022081806-2022082405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022082406-2022083005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022083006-2022090505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022090506-2022091105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022091106-2022091705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022091706-2022092305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022092306-2022092905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022092906-2022100505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022100506-2022101105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022101106-2022101705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022101706-2022102305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022102306-2022102905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022102906-2022110405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022110406-2022111005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022111006-2022111605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022111606-2022112205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022112206-2022112805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022112806-2022120405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022120406-2022121005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022121006-2022121605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022121606-2022122205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022122206-2022122805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2022122806-2023010305_1h_step.nc\n"
     ]
    }
   ],
   "source": [
    "gc.collect()\n",
    "\n",
    "start = '2022-01-02 06:00'\n",
    "end = '2023-01-03 05:00'\n",
    "\n",
    "freq = '1h'\n",
    "fxx_range = range(0,1)\n",
    "query_len_days = 6\n",
    "save_dir='nc_files/hrrr/'\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs')\n",
    "\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled')\n",
    "p_ac_cs_all = p_ac_cs_all.reset_index().set_index('time_center_labeled')\n",
    "\n",
    "ds.to_netcdf('nc_files/ds_act_2022.nc')\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_act_2022.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loaded hrrr_solar_f0-f0_1h_2023010306-2023010905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023010906-2023011505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023011506-2023012105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023012106-2023012705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023012706-2023020205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023020206-2023020805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023020806-2023021405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023021406-2023022005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023022006-2023022605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023022606-2023030405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023030406-2023031005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023031006-2023031605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023031606-2023032205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023032206-2023032805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023032806-2023040305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023040306-2023040905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023040906-2023041505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023041506-2023042105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023042106-2023042705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023042706-2023050305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023050306-2023050905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023050906-2023051505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023051506-2023052105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023052106-2023052705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023052706-2023060205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023060206-2023060805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023060806-2023061405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023061406-2023062005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023062006-2023062605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023062606-2023070205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023070206-2023070805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023070806-2023071405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023071406-2023072005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023072006-2023072605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023072606-2023080105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023080106-2023080705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023080706-2023081305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023081306-2023081905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023081906-2023082505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023082506-2023083105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023083106-2023090605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023090606-2023091205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023091206-2023091805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023091806-2023092405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023092406-2023093005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023093006-2023100605_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023100606-2023101205_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023101206-2023101805_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023101806-2023102405_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023102406-2023103005_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023103006-2023110505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023110506-2023111105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023111106-2023111705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023111706-2023112305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023112306-2023112905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023112906-2023120505_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023120506-2023121105_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023121106-2023121705_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023121706-2023122305_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023122306-2023122905_1h_step.nc\n",
      "loaded hrrr_solar_f0-f0_1h_2023122906-2024010405_1h_step.nc\n"
     ]
    }
   ],
   "source": [
    "gc.collect()\n",
    "\n",
    "start = '2023-01-03 06:00'\n",
    "end = '2024-01-04 05:00'\n",
    "\n",
    "freq = '1h'\n",
    "fxx_range = range(0,1)\n",
    "query_len_days = 6\n",
    "save_dir='nc_files/hrrr/'\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs')\n",
    "\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled')\n",
    "p_ac_cs_all = p_ac_cs_all.reset_index().set_index('time_center_labeled')\n",
    "\n",
    "ds.to_netcdf('nc_files/ds_act_2023.nc')\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_act_2023.pkl')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, get HRRR day ahead forecasts for 2021, 2022, and 2023:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loaded hrrr_solar_f27-f44_1h_2020123106-2021010206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021010306-2021010506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021010606-2021010806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021010906-2021011106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021011206-2021011406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021011506-2021011706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021011806-2021012006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021012106-2021012306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021012406-2021012606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021012706-2021012906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021013006-2021020106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021020206-2021020406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021020506-2021020706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021020806-2021021006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021021106-2021021306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021021406-2021021606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021021706-2021021906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021022006-2021022206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021022306-2021022506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021022606-2021022806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021030106-2021030306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021030406-2021030606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021030706-2021030906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031006-2021031206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031306-2021031506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031606-2021031806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031906-2021032106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021032206-2021032406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021032506-2021032706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021032806-2021033006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021033106-2021040206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021040306-2021040506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021040606-2021040806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021040906-2021041106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021041206-2021041406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021041506-2021041706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021041806-2021042006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021042106-2021042306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021042406-2021042606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021042706-2021042906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021043006-2021050206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021050306-2021050506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021050606-2021050806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021050906-2021051106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021051206-2021051406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021051506-2021051706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021051806-2021052006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021052106-2021052306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021052406-2021052606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021052706-2021052906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021053006-2021060106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021060206-2021060406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021060506-2021060706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021060806-2021061006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021061106-2021061306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021061406-2021061606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021061706-2021061906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062006-2021062206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062306-2021062506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062606-2021062806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062906-2021070106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021070206-2021070406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021070506-2021070706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021070806-2021071006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021071106-2021071306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021071406-2021071606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021071706-2021071906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072006-2021072206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072306-2021072506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072606-2021072806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072906-2021073106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021080106-2021080306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021080406-2021080606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021080706-2021080906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081006-2021081206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081306-2021081506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081606-2021081806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081906-2021082106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021082206-2021082406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021082506-2021082706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021082806-2021083006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021083106-2021090206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021090306-2021090506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021090606-2021090806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021090906-2021091106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021091206-2021091406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021091506-2021091706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021091806-2021092006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021092106-2021092306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021092406-2021092606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021092706-2021092906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021093006-2021100206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021100306-2021100506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021100606-2021100806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021100906-2021101106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021101206-2021101406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021101506-2021101706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021101806-2021102006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021102106-2021102306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021102406-2021102606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021102706-2021102906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021103006-2021110106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021110206-2021110406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021110506-2021110706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021110806-2021111006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021111106-2021111306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021111406-2021111606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021111706-2021111906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112006-2021112206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112306-2021112506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112606-2021112806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112906-2021120106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021120206-2021120406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021120506-2021120706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021120806-2021121006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021121106-2021121306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021121406-2021121606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021121706-2021121906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122006-2021122206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122306-2021122506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122606-2021122806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122906-2021123106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2020123106-2021010206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021010306-2021010506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021010606-2021010806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021010906-2021011106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021011206-2021011406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021011506-2021011706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021011806-2021012006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021012106-2021012306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021012406-2021012606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021012706-2021012906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021013006-2021020106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021020206-2021020406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021020506-2021020706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021020806-2021021006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021021106-2021021306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021021406-2021021606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021021706-2021021906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021022006-2021022206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021022306-2021022506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021022606-2021022806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021030106-2021030306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021030406-2021030606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021030706-2021030906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031006-2021031206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031306-2021031506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031606-2021031806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031906-2021032106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021032206-2021032406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021032506-2021032706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021032806-2021033006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021033106-2021040206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021040306-2021040506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021040606-2021040806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021040906-2021041106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021041206-2021041406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021041506-2021041706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021041806-2021042006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021042106-2021042306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021042406-2021042606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021042706-2021042906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021043006-2021050206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021050306-2021050506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021050606-2021050806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021050906-2021051106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021051206-2021051406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021051506-2021051706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021051806-2021052006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021052106-2021052306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021052406-2021052606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021052706-2021052906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021053006-2021060106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021060206-2021060406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021060506-2021060706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021060806-2021061006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021061106-2021061306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021061406-2021061606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021061706-2021061906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062006-2021062206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062306-2021062506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062606-2021062806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062906-2021070106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021070206-2021070406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021070506-2021070706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021070806-2021071006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021071106-2021071306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021071406-2021071606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021071706-2021071906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072006-2021072206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072306-2021072506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072606-2021072806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072906-2021073106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021080106-2021080306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021080406-2021080606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021080706-2021080906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081006-2021081206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081306-2021081506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081606-2021081806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081906-2021082106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021082206-2021082406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021082506-2021082706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021082806-2021083006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021083106-2021090206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021090306-2021090506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021090606-2021090806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021090906-2021091106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021091206-2021091406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021091506-2021091706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021091806-2021092006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021092106-2021092306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021092406-2021092606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021092706-2021092906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021093006-2021100206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021100306-2021100506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021100606-2021100806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021100906-2021101106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021101206-2021101406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021101506-2021101706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021101806-2021102006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021102106-2021102306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021102406-2021102606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021102706-2021102906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021103006-2021110106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021110206-2021110406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021110506-2021110706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021110806-2021111006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021111106-2021111306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021111406-2021111606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021111706-2021111906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112006-2021112206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112306-2021112506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112606-2021112806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112906-2021120106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021120206-2021120406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021120506-2021120706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021120806-2021121006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021121106-2021121306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021121406-2021121606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021121706-2021121906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122006-2021122206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122306-2021122506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122606-2021122806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122906-2021123106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2020123106-2021010206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021010306-2021010506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021010606-2021010806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021010906-2021011106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021011206-2021011406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021011506-2021011706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021011806-2021012006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021012106-2021012306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021012406-2021012606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021012706-2021012906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021013006-2021020106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021020206-2021020406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021020506-2021020706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021020806-2021021006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021021106-2021021306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021021406-2021021606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021021706-2021021906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021022006-2021022206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021022306-2021022506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021022606-2021022806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021030106-2021030306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021030406-2021030606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021030706-2021030906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031006-2021031206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031306-2021031506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031606-2021031806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021031906-2021032106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021032206-2021032406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021032506-2021032706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021032806-2021033006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021033106-2021040206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021040306-2021040506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021040606-2021040806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021040906-2021041106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021041206-2021041406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021041506-2021041706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021041806-2021042006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021042106-2021042306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021042406-2021042606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021042706-2021042906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021043006-2021050206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021050306-2021050506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021050606-2021050806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021050906-2021051106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021051206-2021051406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021051506-2021051706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021051806-2021052006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021052106-2021052306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021052406-2021052606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021052706-2021052906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021053006-2021060106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021060206-2021060406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021060506-2021060706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021060806-2021061006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021061106-2021061306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021061406-2021061606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021061706-2021061906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062006-2021062206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062306-2021062506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062606-2021062806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021062906-2021070106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021070206-2021070406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021070506-2021070706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021070806-2021071006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021071106-2021071306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021071406-2021071606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021071706-2021071906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072006-2021072206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072306-2021072506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072606-2021072806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021072906-2021073106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021080106-2021080306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021080406-2021080606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021080706-2021080906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081006-2021081206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081306-2021081506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081606-2021081806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021081906-2021082106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021082206-2021082406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021082506-2021082706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021082806-2021083006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021083106-2021090206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021090306-2021090506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021090606-2021090806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021090906-2021091106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021091206-2021091406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021091506-2021091706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021091806-2021092006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021092106-2021092306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021092406-2021092606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021092706-2021092906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021093006-2021100206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021100306-2021100506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021100606-2021100806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021100906-2021101106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021101206-2021101406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021101506-2021101706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021101806-2021102006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021102106-2021102306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021102406-2021102606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021102706-2021102906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021103006-2021110106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021110206-2021110406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021110506-2021110706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021110806-2021111006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021111106-2021111306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021111406-2021111606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021111706-2021111906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112006-2021112206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112306-2021112506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112606-2021112806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021112906-2021120106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021120206-2021120406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021120506-2021120706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021120806-2021121006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021121106-2021121306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021121406-2021121606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021121706-2021121906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122006-2021122206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122306-2021122506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122606-2021122806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2021122906-2021123106_24h_step.nc\n"
     ]
    }
   ],
   "source": [
    "start = '2020-12-31T06:00:00'\n",
    "end = '2021-12-31T06:00'\n",
    "\n",
    "freq = '24h'\n",
    "fxx_range = range(27,45)\n",
    "query_len_days = 3\n",
    "\n",
    "save_dir = 'nc_files/hrrr/'\n",
    "remove_grib=True\n",
    "\n",
    "# coarsened mean\n",
    "gc.collect()\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs',\n",
    "                                    coarsen_type='mean', window_size=10)\n",
    "ds.to_netcdf('nc_files/ds_fcast_2021.nc')\n",
    "ds.close()\n",
    "del ds\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_fcast_2021.pkl')\n",
    "\n",
    "# save a clear sky profile \n",
    "p_ac_cs_all = p_ac_cs_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_cs_all.to_pickle('data_files/p_ac_all_cs_2021.pkl')\n",
    "\n",
    "# coarsened max\n",
    "gc.collect()\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs',\n",
    "                                    coarsen_type='max', window_size=10)\n",
    "ds.to_netcdf('nc_files/ds_fcast_max_2021.nc')\n",
    "ds.close()\n",
    "del ds\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_fcast_max_2021.pkl')\n",
    "\n",
    "# coarsened min\n",
    "gc.collect()\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs',\n",
    "                                    coarsen_type='min', window_size=10)\n",
    "ds.to_netcdf('nc_files/ds_fcast_min_2021.nc')\n",
    "ds.close()\n",
    "del ds\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_fcast_min_2021.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loaded hrrr_solar_f27-f44_1h_2022010106-2022010306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022010406-2022010606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022010706-2022010906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011006-2022011206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011306-2022011506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011606-2022011806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011906-2022012106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022012206-2022012406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022012506-2022012706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022012806-2022013006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022013106-2022020206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022020306-2022020506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022020606-2022020806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022020906-2022021106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022021206-2022021406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022021506-2022021706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022021806-2022022006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022022106-2022022306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022022406-2022022606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022022706-2022030106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022030206-2022030406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022030506-2022030706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022030806-2022031006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022031106-2022031306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022031406-2022031606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022031706-2022031906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032006-2022032206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032306-2022032506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032606-2022032806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032906-2022033106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022040106-2022040306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022040406-2022040606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022040706-2022040906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041006-2022041206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041306-2022041506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041606-2022041806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041906-2022042106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022042206-2022042406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022042506-2022042706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022042806-2022043006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022050106-2022050306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022050406-2022050606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022050706-2022050906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051006-2022051206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051306-2022051506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051606-2022051806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051906-2022052106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022052206-2022052406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022052506-2022052706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022052806-2022053006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022053106-2022060206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022060306-2022060506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022060606-2022060806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022060906-2022061106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022061206-2022061406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022061506-2022061706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022061806-2022062006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022062106-2022062306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022062406-2022062606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022062706-2022062906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022063006-2022070206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022070306-2022070506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022070606-2022070806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022070906-2022071106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022071206-2022071406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022071506-2022071706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022071806-2022072006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022072106-2022072306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022072406-2022072606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022072706-2022072906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022073006-2022080106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022080206-2022080406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022080506-2022080706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022080806-2022081006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022081106-2022081306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022081406-2022081606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022081706-2022081906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082006-2022082206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082306-2022082506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082606-2022082806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082906-2022083106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022090106-2022090306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022090406-2022090606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022090706-2022090906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091006-2022091206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091306-2022091506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091606-2022091806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091906-2022092106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022092206-2022092406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022092506-2022092706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022092806-2022093006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022100106-2022100306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022100406-2022100606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022100706-2022100906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101006-2022101206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101306-2022101506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101606-2022101806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101906-2022102106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022102206-2022102406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022102506-2022102706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022102806-2022103006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022103106-2022110206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022110306-2022110506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022110606-2022110806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022110906-2022111106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022111206-2022111406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022111506-2022111706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022111806-2022112006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022112106-2022112306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022112406-2022112606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022112706-2022112906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022113006-2022120206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022120306-2022120506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022120606-2022120806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022120906-2022121106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022121206-2022121406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022121506-2022121706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022121806-2022122006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022122106-2022122306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022122406-2022122606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022122706-2022122906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022123006-2023010106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022010106-2022010306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022010406-2022010606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022010706-2022010906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011006-2022011206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011306-2022011506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011606-2022011806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011906-2022012106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022012206-2022012406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022012506-2022012706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022012806-2022013006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022013106-2022020206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022020306-2022020506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022020606-2022020806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022020906-2022021106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022021206-2022021406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022021506-2022021706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022021806-2022022006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022022106-2022022306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022022406-2022022606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022022706-2022030106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022030206-2022030406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022030506-2022030706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022030806-2022031006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022031106-2022031306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022031406-2022031606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022031706-2022031906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032006-2022032206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032306-2022032506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032606-2022032806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032906-2022033106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022040106-2022040306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022040406-2022040606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022040706-2022040906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041006-2022041206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041306-2022041506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041606-2022041806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041906-2022042106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022042206-2022042406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022042506-2022042706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022042806-2022043006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022050106-2022050306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022050406-2022050606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022050706-2022050906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051006-2022051206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051306-2022051506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051606-2022051806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051906-2022052106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022052206-2022052406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022052506-2022052706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022052806-2022053006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022053106-2022060206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022060306-2022060506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022060606-2022060806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022060906-2022061106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022061206-2022061406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022061506-2022061706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022061806-2022062006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022062106-2022062306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022062406-2022062606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022062706-2022062906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022063006-2022070206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022070306-2022070506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022070606-2022070806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022070906-2022071106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022071206-2022071406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022071506-2022071706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022071806-2022072006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022072106-2022072306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022072406-2022072606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022072706-2022072906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022073006-2022080106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022080206-2022080406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022080506-2022080706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022080806-2022081006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022081106-2022081306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022081406-2022081606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022081706-2022081906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082006-2022082206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082306-2022082506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082606-2022082806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082906-2022083106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022090106-2022090306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022090406-2022090606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022090706-2022090906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091006-2022091206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091306-2022091506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091606-2022091806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091906-2022092106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022092206-2022092406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022092506-2022092706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022092806-2022093006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022100106-2022100306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022100406-2022100606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022100706-2022100906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101006-2022101206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101306-2022101506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101606-2022101806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101906-2022102106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022102206-2022102406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022102506-2022102706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022102806-2022103006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022103106-2022110206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022110306-2022110506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022110606-2022110806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022110906-2022111106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022111206-2022111406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022111506-2022111706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022111806-2022112006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022112106-2022112306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022112406-2022112606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022112706-2022112906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022113006-2022120206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022120306-2022120506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022120606-2022120806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022120906-2022121106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022121206-2022121406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022121506-2022121706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022121806-2022122006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022122106-2022122306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022122406-2022122606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022122706-2022122906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022123006-2023010106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022010106-2022010306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022010406-2022010606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022010706-2022010906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011006-2022011206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011306-2022011506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011606-2022011806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022011906-2022012106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022012206-2022012406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022012506-2022012706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022012806-2022013006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022013106-2022020206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022020306-2022020506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022020606-2022020806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022020906-2022021106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022021206-2022021406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022021506-2022021706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022021806-2022022006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022022106-2022022306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022022406-2022022606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022022706-2022030106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022030206-2022030406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022030506-2022030706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022030806-2022031006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022031106-2022031306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022031406-2022031606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022031706-2022031906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032006-2022032206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032306-2022032506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032606-2022032806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022032906-2022033106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022040106-2022040306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022040406-2022040606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022040706-2022040906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041006-2022041206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041306-2022041506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041606-2022041806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022041906-2022042106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022042206-2022042406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022042506-2022042706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022042806-2022043006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022050106-2022050306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022050406-2022050606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022050706-2022050906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051006-2022051206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051306-2022051506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051606-2022051806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022051906-2022052106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022052206-2022052406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022052506-2022052706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022052806-2022053006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022053106-2022060206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022060306-2022060506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022060606-2022060806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022060906-2022061106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022061206-2022061406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022061506-2022061706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022061806-2022062006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022062106-2022062306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022062406-2022062606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022062706-2022062906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022063006-2022070206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022070306-2022070506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022070606-2022070806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022070906-2022071106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022071206-2022071406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022071506-2022071706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022071806-2022072006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022072106-2022072306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022072406-2022072606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022072706-2022072906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022073006-2022080106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022080206-2022080406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022080506-2022080706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022080806-2022081006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022081106-2022081306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022081406-2022081606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022081706-2022081906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082006-2022082206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082306-2022082506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082606-2022082806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022082906-2022083106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022090106-2022090306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022090406-2022090606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022090706-2022090906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091006-2022091206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091306-2022091506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091606-2022091806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022091906-2022092106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022092206-2022092406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022092506-2022092706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022092806-2022093006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022100106-2022100306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022100406-2022100606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022100706-2022100906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101006-2022101206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101306-2022101506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101606-2022101806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022101906-2022102106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022102206-2022102406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022102506-2022102706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022102806-2022103006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022103106-2022110206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022110306-2022110506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022110606-2022110806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022110906-2022111106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022111206-2022111406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022111506-2022111706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022111806-2022112006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022112106-2022112306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022112406-2022112606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022112706-2022112906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022113006-2022120206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022120306-2022120506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022120606-2022120806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022120906-2022121106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022121206-2022121406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022121506-2022121706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022121806-2022122006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022122106-2022122306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022122406-2022122606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022122706-2022122906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2022123006-2023010106_24h_step.nc\n"
     ]
    }
   ],
   "source": [
    "start = '2022-01-01T06:00:00'\n",
    "end = '2023-01-01T06:00'\n",
    "\n",
    "freq = '24h'\n",
    "fxx_range = range(27,45)\n",
    "query_len_days = 3\n",
    "\n",
    "save_dir = 'nc_files/hrrr/'\n",
    "remove_grib=True\n",
    "\n",
    "# coarsened mean\n",
    "gc.collect()\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs',\n",
    "                                    coarsen_type='mean', window_size=10)\n",
    "ds.to_netcdf('nc_files/ds_fcast_2022.nc')\n",
    "ds.close()\n",
    "del ds\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_fcast_2022.pkl')\n",
    "\n",
    "# save a clear sky profile \n",
    "p_ac_cs_all = p_ac_cs_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_cs_all.to_pickle('data_files/p_ac_all_cs_2022.pkl')\n",
    "\n",
    "# coarsened max\n",
    "gc.collect()\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs',\n",
    "                                    coarsen_type='max', window_size=10)\n",
    "ds.to_netcdf('nc_files/ds_fcast_max_2022.nc')\n",
    "ds.close()\n",
    "del ds\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_fcast_max_2022.pkl')\n",
    "\n",
    "# coarsened min\n",
    "gc.collect()\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs',\n",
    "                                    coarsen_type='min', window_size=10)\n",
    "ds.to_netcdf('nc_files/ds_fcast_min_2022.nc')\n",
    "ds.close()\n",
    "del ds\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_fcast_min_2022.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loaded hrrr_solar_f27-f44_1h_2023010206-2023010406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023010506-2023010706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023010806-2023011006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023011106-2023011306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023011406-2023011606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023011706-2023011906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012006-2023012206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012306-2023012506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012606-2023012806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012906-2023013106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023020106-2023020306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023020406-2023020606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023020706-2023020906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021006-2023021206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021306-2023021506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021606-2023021806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021906-2023022106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023022206-2023022406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023022506-2023022706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023022806-2023030206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023030306-2023030506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023030606-2023030806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023030906-2023031106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023031206-2023031406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023031506-2023031706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023031806-2023032006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023032106-2023032306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023032406-2023032606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023032706-2023032906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023033006-2023040106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023040206-2023040406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023040506-2023040706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023040806-2023041006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023041106-2023041306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023041406-2023041606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023041706-2023041906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042006-2023042206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042306-2023042506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042606-2023042806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042906-2023050106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023050206-2023050406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023050506-2023050706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023050806-2023051006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023051106-2023051306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023051406-2023051606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023051706-2023051906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052006-2023052206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052306-2023052506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052606-2023052806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052906-2023053106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023060106-2023060306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023060406-2023060606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023060706-2023060906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061006-2023061206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061306-2023061506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061606-2023061806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061906-2023062106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023062206-2023062406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023062506-2023062706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023062806-2023063006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023070106-2023070306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023070406-2023070606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023070706-2023070906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071006-2023071206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071306-2023071506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071606-2023071806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071906-2023072106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023072206-2023072406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023072506-2023072706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023072806-2023073006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023073106-2023080206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023080306-2023080506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023080606-2023080806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023080906-2023081106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023081206-2023081406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023081506-2023081706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023081806-2023082006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023082106-2023082306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023082406-2023082606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023082706-2023082906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023083006-2023090106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023090206-2023090406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023090506-2023090706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023090806-2023091006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023091106-2023091306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023091406-2023091606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023091706-2023091906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092006-2023092206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092306-2023092506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092606-2023092806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092906-2023100106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023100206-2023100406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023100506-2023100706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023100806-2023101006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023101106-2023101306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023101406-2023101606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023101706-2023101906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102006-2023102206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102306-2023102506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102606-2023102806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102906-2023103106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023110106-2023110306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023110406-2023110606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023110706-2023110906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111006-2023111206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111306-2023111506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111606-2023111806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111906-2023112106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023112206-2023112406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023112506-2023112706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023112806-2023113006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023120106-2023120306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023120406-2023120606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023120706-2023120906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121006-2023121206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121306-2023121506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121606-2023121806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121906-2023122106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023122206-2023122406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023122506-2023122706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023122806-2023123006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023123106-2024010206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023010206-2023010406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023010506-2023010706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023010806-2023011006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023011106-2023011306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023011406-2023011606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023011706-2023011906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012006-2023012206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012306-2023012506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012606-2023012806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012906-2023013106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023020106-2023020306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023020406-2023020606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023020706-2023020906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021006-2023021206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021306-2023021506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021606-2023021806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021906-2023022106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023022206-2023022406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023022506-2023022706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023022806-2023030206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023030306-2023030506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023030606-2023030806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023030906-2023031106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023031206-2023031406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023031506-2023031706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023031806-2023032006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023032106-2023032306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023032406-2023032606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023032706-2023032906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023033006-2023040106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023040206-2023040406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023040506-2023040706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023040806-2023041006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023041106-2023041306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023041406-2023041606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023041706-2023041906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042006-2023042206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042306-2023042506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042606-2023042806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042906-2023050106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023050206-2023050406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023050506-2023050706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023050806-2023051006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023051106-2023051306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023051406-2023051606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023051706-2023051906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052006-2023052206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052306-2023052506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052606-2023052806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052906-2023053106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023060106-2023060306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023060406-2023060606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023060706-2023060906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061006-2023061206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061306-2023061506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061606-2023061806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061906-2023062106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023062206-2023062406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023062506-2023062706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023062806-2023063006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023070106-2023070306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023070406-2023070606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023070706-2023070906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071006-2023071206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071306-2023071506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071606-2023071806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071906-2023072106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023072206-2023072406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023072506-2023072706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023072806-2023073006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023073106-2023080206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023080306-2023080506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023080606-2023080806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023080906-2023081106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023081206-2023081406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023081506-2023081706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023081806-2023082006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023082106-2023082306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023082406-2023082606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023082706-2023082906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023083006-2023090106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023090206-2023090406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023090506-2023090706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023090806-2023091006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023091106-2023091306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023091406-2023091606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023091706-2023091906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092006-2023092206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092306-2023092506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092606-2023092806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092906-2023100106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023100206-2023100406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023100506-2023100706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023100806-2023101006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023101106-2023101306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023101406-2023101606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023101706-2023101906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102006-2023102206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102306-2023102506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102606-2023102806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102906-2023103106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023110106-2023110306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023110406-2023110606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023110706-2023110906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111006-2023111206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111306-2023111506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111606-2023111806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111906-2023112106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023112206-2023112406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023112506-2023112706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023112806-2023113006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023120106-2023120306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023120406-2023120606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023120706-2023120906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121006-2023121206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121306-2023121506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121606-2023121806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121906-2023122106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023122206-2023122406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023122506-2023122706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023122806-2023123006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023123106-2024010206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023010206-2023010406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023010506-2023010706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023010806-2023011006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023011106-2023011306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023011406-2023011606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023011706-2023011906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012006-2023012206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012306-2023012506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012606-2023012806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023012906-2023013106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023020106-2023020306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023020406-2023020606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023020706-2023020906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021006-2023021206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021306-2023021506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021606-2023021806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023021906-2023022106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023022206-2023022406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023022506-2023022706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023022806-2023030206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023030306-2023030506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023030606-2023030806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023030906-2023031106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023031206-2023031406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023031506-2023031706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023031806-2023032006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023032106-2023032306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023032406-2023032606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023032706-2023032906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023033006-2023040106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023040206-2023040406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023040506-2023040706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023040806-2023041006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023041106-2023041306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023041406-2023041606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023041706-2023041906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042006-2023042206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042306-2023042506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042606-2023042806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023042906-2023050106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023050206-2023050406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023050506-2023050706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023050806-2023051006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023051106-2023051306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023051406-2023051606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023051706-2023051906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052006-2023052206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052306-2023052506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052606-2023052806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023052906-2023053106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023060106-2023060306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023060406-2023060606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023060706-2023060906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061006-2023061206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061306-2023061506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061606-2023061806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023061906-2023062106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023062206-2023062406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023062506-2023062706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023062806-2023063006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023070106-2023070306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023070406-2023070606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023070706-2023070906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071006-2023071206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071306-2023071506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071606-2023071806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023071906-2023072106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023072206-2023072406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023072506-2023072706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023072806-2023073006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023073106-2023080206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023080306-2023080506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023080606-2023080806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023080906-2023081106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023081206-2023081406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023081506-2023081706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023081806-2023082006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023082106-2023082306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023082406-2023082606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023082706-2023082906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023083006-2023090106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023090206-2023090406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023090506-2023090706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023090806-2023091006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023091106-2023091306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023091406-2023091606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023091706-2023091906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092006-2023092206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092306-2023092506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092606-2023092806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023092906-2023100106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023100206-2023100406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023100506-2023100706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023100806-2023101006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023101106-2023101306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023101406-2023101606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023101706-2023101906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102006-2023102206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102306-2023102506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102606-2023102806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023102906-2023103106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023110106-2023110306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023110406-2023110606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023110706-2023110906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111006-2023111206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111306-2023111506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111606-2023111806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023111906-2023112106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023112206-2023112406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023112506-2023112706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023112806-2023113006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023120106-2023120306_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023120406-2023120606_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023120706-2023120906_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121006-2023121206_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121306-2023121506_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121606-2023121806_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023121906-2023122106_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023122206-2023122406_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023122506-2023122706_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023122806-2023123006_24h_step.nc\n",
      "loaded hrrr_solar_f27-f44_1h_2023123106-2024010206_24h_step.nc\n"
     ]
    }
   ],
   "source": [
    "start = '2023-01-02T06:00:00'\n",
    "end = '2024-01-01T06:00'\n",
    "\n",
    "freq = '24h'\n",
    "fxx_range = range(27,45)\n",
    "query_len_days = 3\n",
    "\n",
    "save_dir = 'nc_files/hrrr/'\n",
    "remove_grib=True\n",
    "\n",
    "# coarsened mean\n",
    "gc.collect()\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs',\n",
    "                                    coarsen_type='mean', window_size=10)\n",
    "ds.to_netcdf('nc_files/ds_fcast_2023.nc')\n",
    "ds.close()\n",
    "del ds\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_fcast_2023.pkl')\n",
    "\n",
    "# save a clear sky profile \n",
    "p_ac_cs_all = p_ac_cs_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_cs_all.to_pickle('data_files/p_ac_all_cs_2023.pkl')\n",
    "\n",
    "# coarsened max\n",
    "gc.collect()\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs',\n",
    "                                    coarsen_type='max', window_size=10)\n",
    "ds.to_netcdf('nc_files/ds_fcast_max_2023.nc')\n",
    "ds.close()\n",
    "del ds\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_fcast_max_2023.pkl')\n",
    "\n",
    "# coarsened min\n",
    "gc.collect()\n",
    "ds, p_ac_all, p_ac_cs_all = get_many_hrrr_fcasts_fastherbie(start, end, freq, query_len_days, fxx_range,\n",
    "                                    plants_df, region_extent=[220,440,1080,1440],\n",
    "                                    save=True, save_dir=save_dir, attempts=5,\n",
    "                                    transposition_model='perez',\n",
    "                                    decomposition_model='erbs',\n",
    "                                    coarsen_type='min', window_size=10)\n",
    "ds.to_netcdf('nc_files/ds_fcast_min_2023.nc')\n",
    "ds.close()\n",
    "del ds\n",
    "p_ac_all = p_ac_all.reset_index().set_index('time_center_labeled').drop(columns={'level_0'})\n",
    "p_ac_all.to_pickle('data_files/p_ac_all_fcast_min_2023.pkl')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can move on to preparing the data for our machine learning models in [08_prep_data_for_machine_learning.ipynb](08_prep_data_for_machine_learning.ipynb)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}