Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add climatological days #25

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions climatological days/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Dataset API use cases

*Responsible: Nicola Pierotti*

How to run the notebooks:

1. Clone the repository to your local machine.
2. Install the requirements.
3. Run the notebooks and feel free to explore more features on meteoblue Dataset API by editing them.



## How to do it?

**1. Clone the repository to your local machine.**
```
cd existing_repo
git clone https://git.meteoblue.com/metoffice/dataset-api-notebooks.git
```

**2. Install and activate a new conda environment**

Let's call this conda environment *usecases_env* (just as an example).
```
conda create --name usecases_env --file requirements.txt
```
```
conda activate usecases_env
```
313 changes: 313 additions & 0 deletions climatological days/consecutive_dry_days.ipynb

Large diffs are not rendered by default.

321 changes: 321 additions & 0 deletions climatological days/frost_days.ipynb

Large diffs are not rendered by default.

321 changes: 321 additions & 0 deletions climatological days/heavy_precipitation_days.ipynb

Large diffs are not rendered by default.

321 changes: 321 additions & 0 deletions climatological days/hot_days.ipynb

Large diffs are not rendered by default.

321 changes: 321 additions & 0 deletions climatological days/ice_days.ipynb

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions climatological days/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
cartopy==0.21.1
dateutil==2.8.2
geopandas==0.13.0
matplotlib==3.5.2
meteoblue_dataset_sdk==1.2.2
numpy==1.21.0
pandas==1.5.0
322 changes: 322 additions & 0 deletions climatological days/tropical_nights.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,322 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "85a41285",
"metadata": {},
"source": [
"# Tropical Nights Map\n",
"\n",
"**Tropical Nights**: daily T > 20°C\n",
"\n",
"Import the meteoblue_dataset_sdk package, which allows to get data from the meteoblue API directly with Python.\n",
"To install the package you can use pip (pip -install meteoblue_dataset_skd), conda or brew."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "05635593",
"metadata": {},
"outputs": [],
"source": [
"import meteoblue_dataset_sdk"
]
},
{
"cell_type": "markdown",
"id": "8e21fbd8",
"metadata": {},
"source": [
"Import several packages which are required for the API call, handling the dataframes and plotting."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "1fb38f4f",
"metadata": {},
"outputs": [],
"source": [
"import datetime as dt\n",
"import dateutil.parser\n",
"import pandas as pd\n",
"import numpy as np\n",
"from cartopy.io import shapereader\n",
"import cartopy.io.img_tiles as cimgt\n",
"import cartopy.crs as ccrs\n",
"import geopandas\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"id": "eac89fcb",
"metadata": {},
"source": [
"Define functions to transform the output of the dataset API into a pandas dataframe"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "b8bdcb7d",
"metadata": {},
"outputs": [],
"source": [
"def meteoblue_timeinterval_to_timestamps(t):\n",
" if len(t.timestrings) > 0:\n",
" def map_ts(time):\n",
" if \"-\" in time:\n",
" return dateutil.parser.parse(time.partition(\"-\")[0])\n",
" return dateutil.parser.parse(time)\n",
"\n",
" return list(map(map_ts, t.timestrings))\n",
"\n",
" timerange = range(t.start, t.end, t.stride)\n",
" return list(map(lambda t: dt.datetime.fromtimestamp(t), timerange))\n",
"\n",
"def meteoblue_result_to_dataframe(geometry):\n",
" t = geometry.timeIntervals[0]\n",
" timestamps = meteoblue_timeinterval_to_timestamps(t)\n",
"\n",
" n_locations = len(geometry.lats)\n",
"\n",
" df = pd.DataFrame(\n",
" {\n",
" \"timestamp\": np.tile(timestamps, n_locations),\n",
" }\n",
" )\n",
" for code in geometry.codes:\n",
" name = str(code.code) + \"_\" + code.level + \"_\" + code.aggregation\n",
" df[name] = list(code.timeIntervals[0].data)\n",
" return df"
]
},
{
"cell_type": "markdown",
"id": "1edfad7b",
"metadata": {},
"source": [
"Select **year** of analysis, **map resolution** (in °) and **apikey**"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "d74e395c",
"metadata": {},
"outputs": [],
"source": [
"apikey = \"XXXXXXXX\"\n",
"year = 2021\n",
"resolution = 0.025 "
]
},
{
"cell_type": "markdown",
"id": "6a06e200",
"metadata": {},
"source": [
"Define the query for **number of yearly tropical nights**"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "ac093e33",
"metadata": {},
"outputs": [],
"source": [
"query_map = {\n",
" \"units\": {\n",
" \"temperature\": \"C\",\n",
" \"velocity\": \"km/h\",\n",
" \"length\": \"metric\",\n",
" \"energy\": \"watts\"\n",
" },\n",
" \"geometry\": {\n",
" \"type\": \"GeonamePolygon\",\n",
" \"geonameid\": 3175395\n",
" },\n",
" \"format\": \"json\",\n",
" \"timeIntervals\": [\n",
" str(year)+\"-01-01T+00:00/\"+str(year)+\"-12-31T+00:00\"\n",
" ],\n",
" \"timeIntervalsAlignment\": \"none\",\n",
" \"queries\": [\n",
" {\n",
" \"domain\": \"ERA5T\",\n",
" \"gapFillDomain\": None,\n",
" \"timeResolution\": \"hourly\",\n",
" \"codes\": [\n",
" {\n",
" \"code\": 193,\n",
" \"level\": \"2 m above gnd\"\n",
" }\n",
" ],\n",
" \"transformations\": [\n",
" {\n",
" \"type\": \"aggregateDaily\",\n",
" \"aggregation\": \"min\"\n",
" },\n",
" {\n",
" \"type\": \"valueIsAbove\",\n",
" \"valueMin\": 20,\n",
" \"returnClassification\": \"zeroOrOne\"\n",
" },\n",
" {\n",
" \"type\": \"aggregateYearly\",\n",
" \"aggregation\": \"sum\"\n",
" },\n",
" {\n",
" \"type\": \"spatialTransform\",\n",
" \"gridResolution\": resolution,\n",
" \"interpolationMethod\": \"linear\",\n",
" \"spatialAggregation\": \"mean\",\n",
" \"disjointArea\": \"discard\",\n",
" \"elevationDownscale\": \"disabled\"\n",
" }\n",
" ]\n",
" }\n",
" ]\n",
"}\n"
]
},
{
"cell_type": "markdown",
"id": "0c7d6b0d",
"metadata": {},
"source": [
"**API call**:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "61eb68d0",
"metadata": {},
"outputs": [],
"source": [
"client = meteoblue_dataset_sdk.Client(apikey)\n",
"\n",
"result = client.query_sync(query_map)\n",
"lats = list(result.geometries[0].lats)\n",
"lons = list(result.geometries[0].lons)\n",
"data = list(meteoblue_result_to_dataframe(result.geometries[0])[\"193_2 m above gnd_sum\"])\n"
]
},
{
"cell_type": "markdown",
"id": "b178d626",
"metadata": {},
"source": [
"Set colormap and scale limits:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "9666f800",
"metadata": {},
"outputs": [],
"source": [
"cmap = \"RdBu_r\"\n",
"min_val = 20\n",
"max_val = 80"
]
},
{
"cell_type": "markdown",
"id": "86b24042",
"metadata": {},
"source": [
"Poduce the plot:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "8f3ca662",
"metadata": {},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'np' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m/var/folders/7b/gkl06hk92qdgj5h3m1qkb1340000gq/T/ipykernel_15542/903421040.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mxs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mys\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmeshgrid\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlons\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlats\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0mdataMesh\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mempty_like\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0md\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlons\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlats\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0mdataMesh\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mlons\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlats\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0md\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mNameError\u001b[0m: name 'np' is not defined"
]
}
],
"source": [
"xs, ys = np.meshgrid(lons, lats)\n",
"dataMesh = np.empty_like(xs)\n",
"for i, j, d in zip(lons, lats, data):\n",
" dataMesh[lons.index(i), lats.index(j)] = d\n",
"\n",
"# request data for use by geopandas\n",
"resolution = '10m'\n",
"category = 'cultural'\n",
"name = 'admin_0_countries'\n",
"\n",
"shpfilename = shapereader.natural_earth(resolution, category, name)\n",
"df = geopandas.read_file(shpfilename)\n",
"\n",
"# get geometry of a country\n",
"poly = [df.loc[df['ADMIN'] == \"Italy\"]['geometry'].values[0]]\n",
"\n",
"stamen_terrain = cimgt.Stamen('terrain-background')\n",
"\n",
"# projections that involved\n",
"st_proj = stamen_terrain.crs #projection used by Stamen images\n",
"ll_proj = ccrs.PlateCarree() #CRS for raw long/lat\n",
"\n",
"# create fig and axes using intended projection\n",
"fig = plt.figure(figsize=(14,12))\n",
"ax = fig.add_subplot(1, 1, 1, projection=st_proj)\n",
"ax.set_title(\"Number of tropical nights for \"+str(year), fontsize = 24)\n",
"ax.add_geometries(poly, crs=ll_proj, facecolor='none', edgecolor='black')\n",
"pad1 = .1 #padding, degrees unit\n",
"exts = [poly[0].bounds[0] - pad1, poly[0].bounds[2] + pad1, poly[0].bounds[1] - pad1, poly[0].bounds[3] + pad1];\n",
"ax.set_extent(exts, crs=ll_proj)\n",
"img=ax.scatter(lons, lats, c=data,s=5, cmap = cmap,vmin=min_val,vmax=max_val, transform =ccrs.PlateCarree())\n",
"color_bar = fig.colorbar(img, ax = ax, extend = 'both', orientation = \"vertical\", fraction = 0.046, pad = 0.04)\n",
"color_bar.ax.tick_params(labelsize=18)\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3d857898",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.13"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading
Loading