This notebook demonstrates MOM6-in-CESM domain nesting: running progressively finer child domains whose open-boundary conditions (OBCs) come from the next coarser parent, rather than from a global reanalysis like GLORYS.
Feature: the Strait of Gibraltar — the narrow (~14 km wide) sill connecting the Atlantic and the Mediterranean. Warm, salty Mediterranean water sinks over the Camarinal Sill (~300 m) and cascades down the Gulf of Cadiz slope as Mediterranean Overflow Water, spinning off eddies (“Meddies”) — a classic hydraulic-control/overflow process that only shows up once resolution is fine enough to resolve the sill itself.
Each level in this hierarchy refines by 4× (the max recommended child:parent ratio) rather than 2×, so the tiny child reaches sub-kilometer, “hyper-resolution” — while every domain stays small enough to build and run on ~10 cores.
Domain hierarchy¶
Parent (gibraltar_parent) 1/12° (~7.5-9 km) -8.0→-2.0 lon, 34.5→37.0 lat (Gulf of Cadiz + Alboran approach)
│ 4× refinement
└── Middle child 1/48° (~1.9-2.3 km) -6.0→-5.0 lon, 35.75→36.25 lat (strait mouth)
│ 4× refinement
└── Tiny child 1/192° (~0.47-0.58 km) -5.9→-5.65 lon, 35.83→35.955 lat (Camarinal Sill)Rule of thumb: each child should be no more than 4× finer than its parent (4:1 ratio) — this hierarchy now sits right at that maximum every step, rather than the more conservative 2× used previously. The parent alone (1/12°) can’t resolve the 14 km-wide strait at all; the middle child (1/48°) resolves the strait mouth with several points across it; the tiny child (1/192°) zooms in tight enough to resolve the Camarinal Sill itself at sub-kilometer resolution.
Workflow¶
The three domains are built up together, step by step, rather than one domain at a time — each step below does the same thing for the parent and both children, since the calls are basically identical across domains:
Paths — case/input paths for all three domains
Grids — horizontal grid for all three (created fresh — there’s no existing reference case for Gibraltar, unlike a production run)
Topography — bathymetry for all three, from GEBCO
Vertical grids — hyperbolic vertical grid for all three
CESM cases — create all three cases, each pinned to a small
NTASKS_OCN(~10 cores or fewer)Forcings — configure OBCs for all three: parent from GLORYS, children from
CESM_MOM_OUTPUT.get_mom6_output_data()reading their parent’s nesting output
Setup — imports¶
%load_ext autoreload
%autoreload 2
from pathlib import Path
from CrocoDash.grid import Grid
from CrocoDash.topo import Topo
from CrocoDash.vgrid import VGrid
from CrocoDash.case import Case
Step 1 — Paths (parent, middle child, tiny child)¶
CESMROOT = Path("~/work/installs/CROCESM_workshop_2025").expanduser()
PROJECT = "ncgd0011"
GEBCO = Path("/glade/derecho/scratch/manishrv/ice/croc/gebco/GEBCO_2024.nc")
# --- Parent (Gulf of Cadiz + Alboran approach, 1/12°) ---
PARENT_CASE = Path("~/croc_cases/gibraltar_parent").expanduser()
PARENT_INPUT = Path("/glade/u/home/manishrv/scratch/croc_input/gibraltar_parent")
# --- Middle child (strait mouth, 1/24°) ---
MID_CASE = Path("~/croc_cases/gibraltar_mid_child").expanduser()
MID_INPUT = Path("/glade/u/home/manishrv/scratch/croc_input/gibraltar_mid_child")
# --- Tiny child (Camarinal Sill / Tarifa Narrows, 1/48°) ---
TINY_CASE = Path("~/croc_cases/gibraltar_tiny_child").expanduser()
TINY_INPUT = Path("/glade/u/home/manishrv/scratch/croc_input/gibraltar_tiny_child")
# --- Nesting sources (each child's OBC comes from the next coarser domain's own MOM6 output) ---
# See the note in Step 6 for what this directory should actually contain -- it's not just
# "point this at wherever the parent archives its history output."
PARENT_NESTING_OUTPUT = Path("~/scratch/archive/gibraltar_parent/ocn/hist/z_files").expanduser()
MID_NESTING_OUTPUT = Path("~/scratch/archive/gibraltar_mid_child/ocn/hist/z_files").expanduser()
Step 2 — Horizontal grids (parent, middle child, tiny child)¶
# Parent — 1/12°, Gulf of Cadiz + western Alboran Sea approach to the strait
parent_grid = Grid(
resolution = 1/12,
xstart = -8.0,
lenx = 6.0, # -8.0 → -2.0
ystart = 34.5,
leny = 2.5, # 34.5 → 37.0
name = "gibraltar_parent",
)
# Middle child — 1/48°, 4× refinement over the parent, zoomed on the strait mouth
mid_grid = Grid(
resolution = 1/48,
xstart = -6.0,
lenx = 1.0, # -6.0 → -5.0
ystart = 35.75,
leny = 0.5, # 35.75 → 36.25
name = "gibraltar_mid_child",
)
# Tiny child — 1/192°, 4× refinement over the middle child, zoomed on the Camarinal Sill
tiny_grid = Grid(
resolution = 1/192,
xstart = -5.9,
lenx = 0.25, # -5.9 → -5.65
ystart = 35.83,
leny = 0.125, # 35.83 → 35.955
name = "gibraltar_tiny_child",
)
print(f"Parent grid: {parent_grid}")
print(f"Middle child grid: {mid_grid}")
print(f"Tiny child grid: {tiny_grid}")
Parent grid: <mom6_forge.grid.Grid object at 0x7f2e38c1a690>
Middle child grid: <mom6_forge.grid.Grid object at 0x7f2e38c1a610>
Tiny child grid: <mom6_forge.grid.Grid object at 0x7f2e38c1b850>
Step 3 — Bathymetry / topography (parent, middle child, tiny child)¶
# Parent — from GEBCO
parent_topo = Topo(grid=parent_grid, min_depth=10.0)
parent_topo.set_from_dataset(
bathymetry_path = GEBCO,
longitude_coordinate_name = "lon",
latitude_coordinate_name = "lat",
vertical_coordinate_name = "elevation",
)
parent_topo.depth.plot(figsize=(8, 4))
print(f"Parent topo depth range: {float(parent_topo.depth.min()):.1f} – {float(parent_topo.depth.max()):.1f} m")
# Middle child — from GEBCO
mid_topo = Topo(grid=mid_grid, min_depth=10.0)
mid_topo.set_from_dataset(
bathymetry_path = GEBCO,
longitude_coordinate_name = "lon",
latitude_coordinate_name = "lat",
vertical_coordinate_name = "elevation",
)
mid_topo.depth.plot(figsize=(6, 4))
# Tiny child — from GEBCO; at 1/192° this should clearly resolve the Camarinal Sill (~300 m)
# rising out of the deeper strait channel on either side
tiny_topo = Topo(grid=tiny_grid, min_depth=10.0)
tiny_topo.set_from_dataset(
bathymetry_path = GEBCO,
longitude_coordinate_name = "lon",
latitude_coordinate_name = "lat",
vertical_coordinate_name = "elevation",
)
tiny_topo.depth.plot(figsize=(6, 4))
**NOTE**
If bathymetry setup fails (e.g. kernel crashes), restart the kernel and edit this cell.
Call ``[topo_object_name].mpi_set_from_dataset()`` instead. Follow the given instructions for using mpi
and ESMF_Regrid outside of a python environment. This breaks up the process, so be sure to call
``[topo_object_name].tidy_dataset() after regridding with mpi.
Slicing source bathymetry to domain: (-8.0, -2.0) x (34.5, 37.0) with buffer 0.5
Begin regridding dataset...
Original dataset size: 2.84 Mb
Regridded size: 0.07 Mb
Generating regridding weights using xESMF with method 'bilinear'...
Tidy bathymetry: Reading in regridded bathymetry to fix up metadata...done. Filling in inland lakes and channels... Parent topo depth range: 0.0 – 2787.0 m
**NOTE**
If bathymetry setup fails (e.g. kernel crashes), restart the kernel and edit this cell.
Call ``[topo_object_name].mpi_set_from_dataset()`` instead. Follow the given instructions for using mpi
and ESMF_Regrid outside of a python environment. This breaks up the process, so be sure to call
``[topo_object_name].tidy_dataset() after regridding with mpi.
Slicing source bathymetry to domain: (-6.0, -5.0) x (35.75, 36.25) with buffer 0.5
Begin regridding dataset...
Original dataset size: 0.35 Mb
Regridded size: 0.04 Mb
Generating regridding weights using xESMF with method 'bilinear'...
Tidy bathymetry: Reading in regridded bathymetry to fix up metadata...done. Filling in inland lakes and channels... **NOTE**
If bathymetry setup fails (e.g. kernel crashes), restart the kernel and edit this cell.
Call ``[topo_object_name].mpi_set_from_dataset()`` instead. Follow the given instructions for using mpi
and ESMF_Regrid outside of a python environment. This breaks up the process, so be sure to call
``[topo_object_name].tidy_dataset() after regridding with mpi.
Slicing source bathymetry to domain: (-5.9, -5.65) x (35.83, 35.955) with buffer 0.5
Begin regridding dataset...
Original dataset size: 0.17 Mb
Regridded size: 0.04 Mb
Generating regridding weights using xESMF with method 'bilinear'...
Tidy bathymetry: Reading in regridded bathymetry to fix up metadata...done. Filling in inland lakes and channels... 
Step 4 — Vertical grids (parent, middle child, tiny child)¶
# Parent — 20 levels, hyperbolic stretching
parent_vgrid = VGrid.hyperbolic(
nk = 20,
depth = float(parent_topo.depth.max()),
ratio = 10.0,
)
# Middle child — 30 levels, hyperbolic stretching
mid_vgrid = VGrid.hyperbolic(
nk = 30,
depth = float(mid_topo.depth.max()),
ratio = 10.0,
)
# Tiny child — 40 levels, hyperbolic stretching (finer near-surface resolution to capture the overflow layer)
tiny_vgrid = VGrid.hyperbolic(
nk = 40,
depth = float(tiny_topo.depth.max()),
ratio = 10.0,
)
Step 5 — Create the CESM cases (parent, middle child, tiny child)¶
All three grids are small enough that NTASKS_OCN is pinned to 10 cores or fewer — this
demo is meant to build and run quickly, not to scale up.
COMPSET = "1850_DATM%NYF_SLND_SICE_MOM6%REGIONAL_SROF_SGLC_SWAV"
parent_case = Case(
cesmroot = CESMROOT,
caseroot = PARENT_CASE,
inputdir = PARENT_INPUT,
compset = COMPSET,
ocn_grid = parent_grid,
ocn_topo = parent_topo,
ocn_vgrid = parent_vgrid,
machine = "derecho",
project = PROJECT,
ntasks_ocn = 10,
override = True,
atm_grid_name = "T62"
)
mid_case = Case(
cesmroot = CESMROOT,
caseroot = MID_CASE,
inputdir = MID_INPUT,
compset = COMPSET,
ocn_grid = mid_grid,
ocn_topo = mid_topo,
ocn_vgrid = mid_vgrid,
machine = "derecho",
project = PROJECT,
ntasks_ocn = 8,
override = True,
atm_grid_name = "T62"
)
tiny_case = Case(
cesmroot = CESMROOT,
caseroot = TINY_CASE,
inputdir = TINY_INPUT,
compset = COMPSET,
ocn_grid = tiny_grid,
ocn_topo = tiny_topo,
ocn_vgrid = tiny_vgrid,
machine = "derecho",
project = PROJECT,
ntasks_ocn = 6,
override = True,
atm_grid_name = "T62"
)
INFO: initialize:Initializing the visualCaseGen system...
INFO: csp_solver:CspSolver initialized.
Creating case...
• Updating ccs_config/modelgrid_aliases_nuopc.xml file to include the new resolution "gibraltar_parent" consisting of the following component grids.
atm grid: "T62"
lnd grid: "T62"
ocn grid: "gibraltar_parent".
rof grid: "null".
• Updating ccs_config/component_grids_nuopc.xml file to include newly generated ocean grid "gibraltar_parent" with the following properties:
nx: 72, ny: 30. ocean mesh: /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/ocnice/ESMF_mesh_gibraltar_parent_757be5.nc.
Running the create_newcase tool with the following command:
/glade/u/home/manishrv/work/installs/CROCESM_workshop_2025/cime/scripts/create_newcase --compset 1850_DATM%NYF_SLND_SICE_MOM6%REGIONAL_SROF_SGLC_SWAV --res gibraltar_parent --case /glade/u/home/manishrv/croc_cases/gibraltar_parent --machine derecho --run-unsupported --project ncgd0011 --non-local
The create_newcase command was successful.
Navigating to the case directory:
cd /glade/u/home/manishrv/croc_cases/gibraltar_parent
Running the case.setup script with the following command:
./case.setup --non-local
Adding parameter changes to user_nl_mom:
! Custom Horizonal Grid, Topography, and Vertical Grid
INPUTDIR = /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/ocnice
TRIPOLAR_N = False
REENTRANT_X = False
REENTRANT_Y = False
NIGLOBAL = 72
NJGLOBAL = 30
GRID_CONFIG = mosaic
GRID_FILE = ocean_hgrid_gibraltar_parent_757be5.nc
TOPO_CONFIG = file
TOPO_FILE = ocean_topog_gibraltar_parent_757be5.nc
MAXIMUM_DEPTH = 2787.0
MINIMUM_DEPTH = 10.0
NK = 20
COORD_CONFIG = none
ALE_COORDINATE_CONFIG = FILE:ocean_vgrid_gibraltar_parent_757be5.nc
REGRIDDING_COORDINATE_MODE = Z*
! Timesteps (based on grid resolution)
DT = 600.0
DT_THERM = 1800.0
INFO: stage:SUCCESS: All stages are complete.
Case created successfully at /glade/u/home/manishrv/croc_cases/gibraltar_parent.
To further customize, build, and run the case, navigate to the case directory in your terminal. To create another case, restart the notebook.
./xmlchange MOM6_MEMORY_MODE=dynamic_symmetric --non-local
./xmlchange NTASKS_OCN=10 --non-local
INFO: initialize:Initializing the visualCaseGen system...
INFO: csp_solver:CspSolver initialized.
Creating case...
• Updating ccs_config/modelgrid_aliases_nuopc.xml file to include the new resolution "gibraltar_mid_child" consisting of the following component grids.
atm grid: "T62"
lnd grid: "T62"
ocn grid: "gibraltar_mid_child".
rof grid: "null".
• Updating ccs_config/component_grids_nuopc.xml file to include newly generated ocean grid "gibraltar_mid_child" with the following properties:
nx: 48, ny: 24. ocean mesh: /glade/u/home/manishrv/scratch/croc_input/gibraltar_mid_child/ocnice/ESMF_mesh_gibraltar_mid_child_79ab1d.nc.
Running the create_newcase tool with the following command:
/glade/u/home/manishrv/work/installs/CROCESM_workshop_2025/cime/scripts/create_newcase --compset 1850_DATM%NYF_SLND_SICE_MOM6%REGIONAL_SROF_SGLC_SWAV --res gibraltar_mid_child --case /glade/u/home/manishrv/croc_cases/gibraltar_mid_child --machine derecho --run-unsupported --project ncgd0011 --non-local
The create_newcase command was successful.
Navigating to the case directory:
cd /glade/u/home/manishrv/croc_cases/gibraltar_mid_child
Running the case.setup script with the following command:
./case.setup --non-local
Adding parameter changes to user_nl_mom:
! Custom Horizonal Grid, Topography, and Vertical Grid
INPUTDIR = /glade/u/home/manishrv/scratch/croc_input/gibraltar_mid_child/ocnice
TRIPOLAR_N = False
REENTRANT_X = False
REENTRANT_Y = False
NIGLOBAL = 48
NJGLOBAL = 24
GRID_CONFIG = mosaic
GRID_FILE = ocean_hgrid_gibraltar_mid_child_79ab1d.nc
TOPO_CONFIG = file
TOPO_FILE = ocean_topog_gibraltar_mid_child_79ab1d.nc
MAXIMUM_DEPTH = 972.0
MINIMUM_DEPTH = 10.0
NK = 30
COORD_CONFIG = none
ALE_COORDINATE_CONFIG = FILE:ocean_vgrid_gibraltar_mid_child_79ab1d.nc
REGRIDDING_COORDINATE_MODE = Z*
! Timesteps (based on grid resolution)
DT = 150.0
DT_THERM = 600.0
INFO: stage:SUCCESS: All stages are complete.
Case created successfully at /glade/u/home/manishrv/croc_cases/gibraltar_mid_child.
To further customize, build, and run the case, navigate to the case directory in your terminal. To create another case, restart the notebook.
./xmlchange MOM6_MEMORY_MODE=dynamic_symmetric --non-local
./xmlchange NTASKS_OCN=8 --non-local
INFO: initialize:Initializing the visualCaseGen system...
INFO: csp_solver:CspSolver initialized.
Creating case...
• Updating ccs_config/modelgrid_aliases_nuopc.xml file to include the new resolution "gibraltar_tiny_child" consisting of the following component grids.
atm grid: "T62"
lnd grid: "T62"
ocn grid: "gibraltar_tiny_child".
rof grid: "null".
• Updating ccs_config/component_grids_nuopc.xml file to include newly generated ocean grid "gibraltar_tiny_child" with the following properties:
nx: 48, ny: 24. ocean mesh: /glade/u/home/manishrv/scratch/croc_input/gibraltar_tiny_child/ocnice/ESMF_mesh_gibraltar_tiny_child_7d7335.nc.
Running the create_newcase tool with the following command:
/glade/u/home/manishrv/work/installs/CROCESM_workshop_2025/cime/scripts/create_newcase --compset 1850_DATM%NYF_SLND_SICE_MOM6%REGIONAL_SROF_SGLC_SWAV --res gibraltar_tiny_child --case /glade/u/home/manishrv/croc_cases/gibraltar_tiny_child --machine derecho --run-unsupported --project ncgd0011 --non-local
The create_newcase command was successful.
Navigating to the case directory:
cd /glade/u/home/manishrv/croc_cases/gibraltar_tiny_child
Running the case.setup script with the following command:
./case.setup --non-local
Adding parameter changes to user_nl_mom:
! Custom Horizonal Grid, Topography, and Vertical Grid
INPUTDIR = /glade/u/home/manishrv/scratch/croc_input/gibraltar_tiny_child/ocnice
TRIPOLAR_N = False
REENTRANT_X = False
REENTRANT_Y = False
NIGLOBAL = 48
NJGLOBAL = 24
GRID_CONFIG = mosaic
GRID_FILE = ocean_hgrid_gibraltar_tiny_child_7d7335.nc
TOPO_CONFIG = file
TOPO_FILE = ocean_topog_gibraltar_tiny_child_7d7335.nc
MAXIMUM_DEPTH = 671.0
MINIMUM_DEPTH = 10.0
NK = 40
COORD_CONFIG = none
ALE_COORDINATE_CONFIG = FILE:ocean_vgrid_gibraltar_tiny_child_7d7335.nc
REGRIDDING_COORDINATE_MODE = Z*
! Timesteps (based on grid resolution)
DT = 25.0
DT_THERM = 100.0
INFO: stage:SUCCESS: All stages are complete.
Case created successfully at /glade/u/home/manishrv/croc_cases/gibraltar_tiny_child.
To further customize, build, and run the case, navigate to the case directory in your terminal. To create another case, restart the notebook.
./xmlchange MOM6_MEMORY_MODE=dynamic_symmetric --non-local
./xmlchange NTASKS_OCN=6 --non-local
Step 6 — Configure and process forcings (parent, middle child, tiny child)¶
The parent gets its OBCs from GLORYS (there’s no existing reference case to reuse for Gibraltar,
unlike a production run). Both children instead read their parent’s nesting output as OBCs,
via CESM_MOM_OUTPUT.get_mom6_output_data.
PARENT_DATES = ["2000-01-01 00:00:00", "2000-01-15 00:00:00"]
MID_DATES = ["2000-01-01 00:00:00", "2000-01-15 00:00:00"]
TINY_DATES = ["2000-01-01 00:00:00", "2000-01-15 00:00:00"]
parent_case.configure_forcings(date_range=PARENT_DATES, function_name = "get_glorys_data_from_rda")
# dataset_path is CESM_MOM_OUTPUT's one non-required arg with no universal default (it's
# meant to point at any user-supplied directory), so each child passes its own parent's
# nesting-output directory in via function_overrides -- see the note above for what that
# directory should contain. function_overrides is written straight to config.json, so its
# values must be JSON-serializable (plain str), not a Path.
mid_case.configure_forcings(
date_range = MID_DATES,
product_name = "cesm_mom_output",
function_name = "get_mom6_output_data",
function_overrides = {"dataset_path": PARENT_NESTING_OUTPUT.as_posix()},
)
tiny_case.configure_forcings(
date_range = TINY_DATES,
product_name = "cesm_mom_output",
function_name = "get_mom6_output_data",
function_overrides = {"dataset_path": MID_NESTING_OUTPUT.as_posix()},
)
parent_case.process_forcings()INFO:GLORYS:Downloading Glorys data from RDA to /glade/derecho/scratch/manishrv/tmp/tmp7wa7xpt9/test_file.notreal
INFO:CrocoDash.extract_forcings.get_dataset_piecewise:[GLORYS.get_glorys_data_from_rda] Usage: Requires read access to /glade/campaign/collections/rda/data/d010049/ and the CrocoDash conda environment.
INFO:CrocoDash.extract_forcings.get_dataset_piecewise:Downloading GLORYS data using get_glorys_data_from_rda from 2000-01-01 00:00:00 to 2000-01-15 00:00:00.
INFO:CrocoDash.extract_forcings.get_dataset_piecewise:Using step size 15, this will result in 1 files per boundary.
INFO:GLORYS:Downloading Glorys data from RDA to /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/extract_forcings/raw_data/ic_unprocessed.nc
INFO:GLORYS:Downloading Glorys data from RDA to /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/extract_forcings/raw_data/south_unprocessed.20000101_20000115.nc
INFO:GLORYS:Downloading Glorys data from RDA to /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/extract_forcings/raw_data/north_unprocessed.20000101_20000115.nc
INFO:GLORYS:Downloading Glorys data from RDA to /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/extract_forcings/raw_data/west_unprocessed.20000101_20000115.nc
INFO:GLORYS:Downloading Glorys data from RDA to /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/extract_forcings/raw_data/east_unprocessed.20000101_20000115.nc
INFO:CrocoDash.extract_forcings.get_dataset_piecewise:Successfully retrieved GLORYS data located in /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/extract_forcings/raw_data directory.
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:Parsing Raw Data Folder
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:All boundaries continuous and non-overlapping.
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:Setting up required information
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:Starting regridding
No arakawa_grid provided, assuming the variable mapping for your data product is already in correct format.
Arakawa A grid detected in variable mapping
INFO:regional_mom6.rotation:Getting rotation angle
INFO:regional_mom6.rotation:Calculating grid rotation angle
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array temp, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array u, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array v, assuming it's in the correct units
WARNING:regional_mom6.regridding:All NaNs filled b/c bathymetry wasn't provided to the function. Add bathymetry_path to the segment class to avoid this
WARNING:regional_mom6.validate:temp_segment_002 does not have a coordinates attribute
WARNING:regional_mom6.validate:salt_segment_002 does not have a coordinates attribute
WARNING:regional_mom6.validate:u_segment_002 does not have a coordinates attribute
WARNING:regional_mom6.validate:v_segment_002 does not have a coordinates attribute
WARNING:regional_mom6.validate:eta_segment_002 does not have a coordinates attribute
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:Saving regridding file as forcing_obc_segment_002_20000101_20000115.nc
INFO:regional_mom6.rotation:Getting rotation angle
INFO:regional_mom6.rotation:Calculating grid rotation angle
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array temp, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array u, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array v, assuming it's in the correct units
WARNING:regional_mom6.regridding:All NaNs filled b/c bathymetry wasn't provided to the function. Add bathymetry_path to the segment class to avoid this
No arakawa_grid provided, assuming the variable mapping for your data product is already in correct format.
Arakawa A grid detected in variable mapping
WARNING:regional_mom6.validate:temp_segment_003 does not have a coordinates attribute
WARNING:regional_mom6.validate:salt_segment_003 does not have a coordinates attribute
WARNING:regional_mom6.validate:u_segment_003 does not have a coordinates attribute
WARNING:regional_mom6.validate:v_segment_003 does not have a coordinates attribute
WARNING:regional_mom6.validate:eta_segment_003 does not have a coordinates attribute
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:Saving regridding file as forcing_obc_segment_003_20000101_20000115.nc
INFO:regional_mom6.rotation:Getting rotation angle
INFO:regional_mom6.rotation:Calculating grid rotation angle
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array temp, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array u, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array v, assuming it's in the correct units
WARNING:regional_mom6.regridding:All NaNs filled b/c bathymetry wasn't provided to the function. Add bathymetry_path to the segment class to avoid this
No arakawa_grid provided, assuming the variable mapping for your data product is already in correct format.
Arakawa A grid detected in variable mapping
WARNING:regional_mom6.validate:temp_segment_004 does not have a coordinates attribute
WARNING:regional_mom6.validate:salt_segment_004 does not have a coordinates attribute
WARNING:regional_mom6.validate:u_segment_004 does not have a coordinates attribute
WARNING:regional_mom6.validate:v_segment_004 does not have a coordinates attribute
WARNING:regional_mom6.validate:eta_segment_004 does not have a coordinates attribute
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:Saving regridding file as forcing_obc_segment_004_20000101_20000115.nc
INFO:regional_mom6.rotation:Getting rotation angle
INFO:regional_mom6.rotation:Calculating grid rotation angle
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array temp, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array u, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array v, assuming it's in the correct units
WARNING:regional_mom6.regridding:All NaNs filled b/c bathymetry wasn't provided to the function. Add bathymetry_path to the segment class to avoid this
No arakawa_grid provided, assuming the variable mapping for your data product is already in correct format.
Arakawa A grid detected in variable mapping
WARNING:regional_mom6.validate:temp_segment_001 does not have a coordinates attribute
WARNING:regional_mom6.validate:salt_segment_001 does not have a coordinates attribute
WARNING:regional_mom6.validate:u_segment_001 does not have a coordinates attribute
WARNING:regional_mom6.validate:v_segment_001 does not have a coordinates attribute
WARNING:regional_mom6.validate:eta_segment_001 does not have a coordinates attribute
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:Saving regridding file as forcing_obc_segment_001_20000101_20000115.nc
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array u, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array v, assuming it's in the correct units
WARNING:regional_mom6.utils:regional_mom6 could not use pint for data array temp, assuming it's in the correct units
Warning: Minimum depth of 4m is less than the depth of the third interface (51.919819898404285m)!
This means that some areas may only have one or two layers between the surface and sea floor.
For increased stability, consider increasing the minimum depth, or adjusting the vertical coordinate to add more layers near the surface.
No arakawa_grid provided, assuming the variable mapping for your data product is already in correct format.
Arakawa A grid detected in variable mapping
INFO:regional_mom6.rotation:Getting rotation angle
INFO:regional_mom6.rotation:Calculating grid rotation angle
WARNING:regional_mom6.validate:eta_t does not have a coordinates attribute
WARNING:regional_mom6.validate:Variable eta_t does not have 4 dimensions
WARNING:regional_mom6.validate:temp does not have a coordinates attribute
WARNING:regional_mom6.validate:Variable temp does not have 4 dimensions
WARNING:regional_mom6.validate:salt does not have a coordinates attribute
WARNING:regional_mom6.validate:Variable salt does not have 4 dimensions
WARNING:regional_mom6.validate:u does not have a coordinates attribute
WARNING:regional_mom6.validate:Variable u does not have 4 dimensions
WARNING:regional_mom6.validate:v does not have a coordinates attribute
WARNING:regional_mom6.validate:Variable v does not have 4 dimensions
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:Start mom6_forge fill...
Setting up Initial Conditions
Regridding Velocities... Done.
Regridding Tracers... Done.
Regridding Free surface... Done.
Saving outputs... INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:...end mom6_forge fill.
INFO:CrocoDash.extract_forcings.regrid_dataset_piecewise:Finished regridding
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:Parsing Regridded Data Folder
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:All boundaries continuous and non-overlapping.
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:Merging Files
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:Saved 001 boundary at /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/ocnice/forcing_obc_segment_001.nc
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:Saved 003 boundary at /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/ocnice/forcing_obc_segment_003.nc
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:Saved 002 boundary at /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/ocnice/forcing_obc_segment_002.nc
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:Saved 004 boundary at /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/ocnice/forcing_obc_segment_004.nc
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:Saved init_eta.nc initial condition to /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/ocnice
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:Saved init_vel.nc initial condition to /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/ocnice
INFO:CrocoDash.extract_forcings.merge_piecewise_dataset:Saved init_tracers.nc initial condition to /glade/u/home/manishrv/scratch/croc_input/gibraltar_parent/ocnice
Case is ready to be built: /glade/u/home/manishrv/croc_cases/gibraltar_parent
Step 7 — Visualize the nested current fields¶
All three domains actually ran to completion (see ~/scratch/archive/gibraltar_*), so we can pull
real surface-current output (speed) and land-mask (geolon/geolat/wet) from each domain’s
mom6.h.sfc and mom6.h.static history files and paint all three directly on top of one another,
in the same map, at each domain’s true geographic location — parent on the bottom, middle child on
top of it, tiny child on top of that — with a thin white outline marking where each child’s
footprint actually starts. One shared color scale across all three, and the displayed window is
cropped in around the strait so the nested children aren’t tiny slivers in the full parent extent.
We also animate it over the run so you can watch the Gibraltar jet meander day by day.
from pathlib import Path
import numpy as np
import xarray as xr
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import matplotlib.patches as mpatches
from matplotlib.colors import Normalize
ARCHIVE = Path("~/scratch/archive").expanduser()
DOMAINS = ["gibraltar_parent", "gibraltar_mid_child", "gibraltar_tiny_child"]
LABELS = {"gibraltar_parent": "1/12°", "gibraltar_mid_child": "1/48°", "gibraltar_tiny_child": "1/192°"}
TRIM = 2 # crop the outermost OBC nudging-zone cells (they host single-cell corner artifacts)
N_FRAMES = 12 # common to all three domains (tiny child is the shortest run)
# crop the displayed window so the nested children aren't tiny slivers in the full parent extent
LON_CROP = (-7.3, -3.3)
LAT_CROP = (34.9, 37.0)
sfc = {d: xr.open_dataset(ARCHIVE / d / "ocn/hist" / f"{d}.mom6.h.sfc.2000-01.nc", decode_timedelta=False) for d in DOMAINS}
static = {d: xr.open_dataset(ARCHIVE / d / "ocn/hist" / f"{d}.mom6.h.static.nc") for d in DOMAINS}
sl2 = (slice(TRIM, -TRIM), slice(TRIM, -TRIM))
geo = {d: dict(lon=static[d].geolon.values[sl2], lat=static[d].geolat.values[sl2],
wet=static[d].wet.values[sl2]) for d in DOMAINS}
all_speed = {d: [] for d in DOMAINS}
for d in DOMAINS:
for t in range(N_FRAMES):
speed = sfc[d].isel(time=t).speed.values[sl2]
all_speed[d].append(np.where(geo[d]["wet"] > 0, speed, np.nan))
# one shared color scale across all three nested domains
pooled = np.concatenate([np.stack(all_speed[d]).ravel() for d in DOMAINS])
norm = Normalize(vmin=0, vmax=np.nanpercentile(pooled, 99))
cmap = plt.get_cmap("RdBu_r")
dates = sfc["gibraltar_parent"].time.values[:N_FRAMES]
def bounds(d):
g = geo[d]
return np.nanmin(g["lon"]), np.nanmax(g["lon"]), np.nanmin(g["lat"]), np.nanmax(g["lat"])
fig, ax = plt.subplots(figsize=(10, 6.5))
# paint each domain directly on top of the last, at its true lon/lat location -- no zoom-box
# callouts, just higher resolution layered over the lower-resolution domain beneath it, with a
# thin white outline marking where each child's footprint actually starts
pms = {}
for i, d in enumerate(DOMAINS):
g = geo[d]
z = i * 2
land = np.where(g["wet"] > 0, np.nan, 1.0)
ax.pcolormesh(g["lon"], g["lat"], land, cmap="Greys", vmin=0, vmax=1.4, shading="auto", zorder=z)
pms[d] = ax.pcolormesh(g["lon"], g["lat"], all_speed[d][0], cmap=cmap, norm=norm, shading="auto", zorder=z + 1)
for d in ["gibraltar_mid_child", "gibraltar_tiny_child"]:
lon0, lon1, lat0, lat1 = bounds(d)
ax.add_patch(mpatches.Rectangle((lon0, lat0), lon1 - lon0, lat1 - lat0,
fill=False, edgecolor="white", linewidth=1.1, zorder=10))
ax.text(np.nanmean(geo["gibraltar_mid_child"]["lon"]), np.nanmax(geo["gibraltar_mid_child"]["lat"]) + 0.02,
LABELS["gibraltar_mid_child"], fontsize=9, fontweight="bold", ha="center", va="bottom", zorder=11)
ax.text(np.nanmean(geo["gibraltar_tiny_child"]["lon"]), np.nanmin(geo["gibraltar_tiny_child"]["lat"]) - 0.015,
LABELS["gibraltar_tiny_child"], fontsize=9, fontweight="bold", ha="center", va="top", zorder=11)
ax.text(LON_CROP[0] + 0.15, LAT_CROP[1] - 0.1, "1/12°", fontsize=9, fontweight="bold", ha="left", va="top")
ax.set_xlim(*LON_CROP)
ax.set_ylim(*LAT_CROP)
ax.set_aspect(1 / np.cos(np.deg2rad(np.mean(LAT_CROP))))
ax.set_xlabel("lon"); ax.set_ylabel("lat")
cb = fig.colorbar(pms["gibraltar_parent"], ax=ax, fraction=0.035, pad=0.02)
cb.set_label("surface current speed (m/s)")
title = ax.set_title("Strait of Gibraltar nested domains")
fig.tight_layout()
out_dir = Path.cwd()
fig.savefig(out_dir / "nesting_hierarchy_viz.png", dpi=150)
plt.show()
# animate over the run so the Gibraltar jet's day-to-day meander is visible
def update(t):
for d in DOMAINS:
pms[d].set_array(all_speed[d][t].ravel())
title.set_text(f"Strait of Gibraltar nested domains — {str(dates[t])[:10]}")
return list(pms.values()) + [title]
anim = animation.FuncAnimation(fig, update, frames=N_FRAMES, blit=False)
anim.save(out_dir / "nesting_hierarchy_movie.gif", writer=animation.PillowWriter(fps=2))
anim.save(out_dir / "nesting_hierarchy_movie.mp4", writer=animation.FFMpegWriter(fps=2, bitrate=3000))
print("saved nesting_hierarchy_viz.png, nesting_hierarchy_movie.gif, nesting_hierarchy_movie.mp4")
saved nesting_hierarchy_viz.png, nesting_hierarchy_movie.gif, nesting_hierarchy_movie.mp4
Summary¶
This notebook walked through a full 3-level nesting hierarchy over the Strait of Gibraltar, building all three domains together at each step, refining by 4× at each level (the max recommended child:parent ratio) so the tiny child reaches sub-kilometer resolution, and kept every domain small enough to build and run on ~10 cores or fewer:
| Domain | Resolution | lon | lat | Grid size | NTASKS_OCN | OBC source |
|---|---|---|---|---|---|---|
| Parent (gibraltar_parent) | 1/12° (~8 km) | -8.0→-2.0 | 34.5→37.0 | ~72×30 | 10 | GLORYS |
| Middle child | 1/48° (~2 km) | -6.0→-5.0 | 35.75→36.25 | ~48×24 | 8 | Parent nesting output |
| Tiny child | 1/192° (~0.5 km) | -5.9→-5.65 | 35.83→35.955 | ~48×24 | 6 | Middle-child nesting output |
Each level runs independently once its case is built. Once the parent finishes, the middle child can start;
once the middle child finishes, the tiny child can start.
See CESM_MOM_OUTPUT (CrocoDash/raw_data_access/datasets/cesm_ocean_output.py) for the
native-MOM6-output readers used above — get_mom6_output_data (multi-variable-per-file
history/diag_table output) and get_mom6_single_variable_data (CESM tseries-style
output) — and their docstrings for direct-call usage. CESM_MOM_OUTPUT is a separate
product from CESM_POP_OUTPUT specifically because they need different variable/
coordinate metadata (native MOM6 names vs. CESM-POP names). Each child’s dataset_path
is threaded through per-call via configure_forcings(function_overrides={"dataset_path": ...})
rather than sourced from class-level metadata, since CESM_MOM_OUTPUT has no single
universal default (it’s meant to point at whichever parent run produced its OBC source) —
see the note in Step 6 for why that directory should hold only the 5 OBC-relevant fields
(zos, thetao, so, uo, vo) rather than a parent’s full history output.
A larger, Caribbean 3-level nesting demo (loading an existing reference case for the
parent instead of building it from scratch) is kept in dev/nesting_caribbean_demo/ for
reference.</cell id=“cell-15”>