cmip6 catalog with 9 dataset(s) from 190 asset(s):
\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
"| \n", " | unique | \n", "
|---|---|
| variable_id | \n", "1 | \n", "
| table_id | \n", "1 | \n", "
| source_id | \n", "9 | \n", "
| experiment_id | \n", "1 | \n", "
| member_id | \n", "4 | \n", "
| grid_label | \n", "3 | \n", "
| time_range | \n", "185 | \n", "
| activity_id | \n", "2 | \n", "
| institution_id | \n", "9 | \n", "
| version | \n", "9 | \n", "
| path | \n", "190 | \n", "
| dcpp_init_year | \n", "0 | \n", "
| derived_variable_id | \n", "0 | \n", "
![\"\"](\"https://upload.wikimedia.org/wikipedia/commons/1/12/Dask_Logo.svg\"){kind=logo}
on the left-side panel.\n",
"```python\n",
"from dask.distributed import Client, LocalCluster\n",
"client = Client(LocalCluster(n_workers=4, threads_per_worker=1, memory_limit='16GB'))\n",
"```\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "3fd710d7",
"metadata": {
"ExecuteTime": {
"end_time": "2024-05-03T12:52:53.705645Z",
"start_time": "2024-05-03T12:52:40.403088Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"--> The keys in the returned dictionary of datasets are constructed as follows:\n",
"\t'activity_id.institution_id'\n"
]
},
{
"data": {
"text/html": [
"\n",
"\n"
],
"text/plain": [
"\n",
" \n",
" 100.00% [9/9 00:13<00:00]\n",
"
\n",
" "
],
"text/plain": [
"\n",
" \n",
" 100.00% [9/9 00:06<00:00]\n",
"
\n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig,ax = plt.subplots(figsize=(10,6))\n",
"for datakey, data in dataset_dict.items(): # Looping over ever varable in the dictionary\n",
" source_id = data.attrs['intake_esm_attrs:source_id']\n",
" member_id = data.attrs['intake_esm_attrs:member_id']\n",
" data['od550aer'].isel(member_id=0).plot(label=f'{source_id} {member_id}', ax=ax)\n",
"\n",
"ax.legend()\n",
" "
]
},
{
"cell_type": "markdown",
"id": "2ecfe25e",
"metadata": {},
"source": [
"### Exporting a subset of the catalog"
]
},
{
"cell_type": "markdown",
"id": "18e01884",
"metadata": {},
"source": [
"Most likely you will only be analyzing a small subset of experiments the model experiment and it could be beneficial to work with a reduced catalog. \n",
"Below we will make a subset that only contain the information related to `histSST` and `histSST-piaer`, and only the absorption optical depth (`od550abs`) and total optical depth (`od550aer`). "
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "d4d015f4",
"metadata": {
"ExecuteTime": {
"end_time": "2024-05-03T12:54:53.492592Z",
"start_time": "2024-05-03T12:54:52.229732Z"
}
},
"outputs": [],
"source": [
"col = intake.open_esm_datastore('/mnt/craas1-ns9989k-geo4992/data/cmip6.json') # Local data stored on NIRD"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "3e57386d",
"metadata": {
"ExecuteTime": {
"end_time": "2024-05-03T12:54:53.645026Z",
"start_time": "2024-05-03T12:54:53.493934Z"
}
},
"outputs": [
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"col = col.search( \n",
" experiment_id=['histSST', 'histSST-piaer'],\n",
" variable_id = ['od550abs', 'od550aer']\n",
")\n",
"col"
]
},
{
"cell_type": "markdown",
"id": "d5678e30",
"metadata": {},
"source": [
"Then when we are happy with the selection the catalog can be exported as follows:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "8131074a",
"metadata": {
"ExecuteTime": {
"end_time": "2024-05-03T12:54:53.652854Z",
"start_time": "2024-05-03T12:54:53.646372Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Successfully wrote ESM catalog json file to: file:///home/fc-3auid-3a9fdc0c87-2d7836-2d4bdc-2db802-2d9a250c322e3b//histSST-AerChemMIP.json\n"
]
}
],
"source": [
"col.serialize(name='histSST-AerChemMIP',catalog_type='file',directory='~/')"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "90b4c58e",
"metadata": {
"ExecuteTime": {
"end_time": "2024-05-03T12:54:53.675734Z",
"start_time": "2024-05-03T12:54:53.654084Z"
}
},
"outputs": [
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"col = intake.open_esm_datastore('~/histSST-AerChemMIP.json')\n",
"col"
]
}
],
"metadata": {
"celltoolbar": "Edit Metadata",
"kernelspec": {
"display_name": "Python [conda env:pangeo-notebook]",
"language": "python",
"name": "conda-env-pangeo-notebook-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 5
}
cmip6 catalog with 9 dataset(s) from 190 asset(s):
\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
"| \n", " | unique | \n", "
|---|---|
| variable_id | \n", "1 | \n", "
| table_id | \n", "1 | \n", "
| source_id | \n", "9 | \n", "
| experiment_id | \n", "1 | \n", "
| member_id | \n", "4 | \n", "
| grid_label | \n", "3 | \n", "
| time_range | \n", "185 | \n", "
| activity_id | \n", "2 | \n", "
| institution_id | \n", "9 | \n", "
| version | \n", "9 | \n", "
| path | \n", "190 | \n", "
| dcpp_init_year | \n", "0 | \n", "
| derived_variable_id | \n", "0 | \n", "
histSST-AerChemMIP catalog with 9 dataset(s) from 190 asset(s):
\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
"| \n", " | unique | \n", "
|---|---|
| variable_id | \n", "1 | \n", "
| table_id | \n", "1 | \n", "
| source_id | \n", "9 | \n", "
| experiment_id | \n", "1 | \n", "
| member_id | \n", "4 | \n", "
| grid_label | \n", "3 | \n", "
| time_range | \n", "185 | \n", "
| activity_id | \n", "2 | \n", "
| institution_id | \n", "9 | \n", "
| version | \n", "9 | \n", "
| path | \n", "190 | \n", "
| dcpp_init_year | \n", "0 | \n", "
| derived_variable_id | \n", "0 | \n", "