Setup the NWA12 domain through CrocoDash#
SECTION 1: Generate a regional MOM6 domain#
We begin by defining a regional MOM6 domain using CrocoDash. To do so, we first carve out a subdomain from a 1/12 degree global grid. We then generate the topography by remapping an existing bathymetric dataset to our horizontal grid. Finally, we define a vertical grid.
Step 1.1: Horizontal Grid#
Extract a subgrid from a global grid using the subgrid_from_supergrid method:
from CrocoDash.grid import Grid
grid = Grid.subgrid_from_supergrid(
path = "s3://crocodile-cesm/CrocoDash/data/grids/ocean_hgrid_trimmed.nc", # supergrid
llc = (5, -99), # (l)ower (l)eft (c)orner coords
urc = (59, -36.), # (u)pper (r)ight (c)orner coords
name = "nwa12"
)
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <?xml^ version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>HMBBCDC6GH5Q2W04</RequestId><HostId>LsjAW1DXoXUoOp6gP6XLQ60hPy8bL/dgjiWmgXQYY7+jvl7IO/iCKHZGgLpB+lFFoW3wA/qlutg=</HostId></Error>
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/file_manager.py:211, in CachingFileManager._acquire_with_cache_info(self, needs_lock)
210 try:
--> 211 file = self._cache[self._key]
212 except KeyError:
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/lru_cache.py:56, in LRUCache.__getitem__(self, key)
55 with self._lock:
---> 56 value = self._cache[key]
57 self._cache.move_to_end(key)
KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('s3://crocodile-cesm/CrocoDash/data/grids/ocean_hgrid_trimmed.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), '97532e44-01c9-437d-9eef-4a20b8b6fc34']
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
Input In [1], in <cell line: 0>()
1 from CrocoDash.grid import Grid
----> 3 grid = Grid.subgrid_from_supergrid(
4 path = "s3://crocodile-cesm/CrocoDash/data/grids/ocean_hgrid_trimmed.nc", # supergrid
5 llc = (5, -99), # (l)ower (l)eft (c)orner coords
6 urc = (59, -36.), # (u)pper (r)ight (c)orner coords
7 name = "nwa12"
8 )
File ~/work/CrocoGallery/CrocoGallery/CrocoDash/CrocoDash/visualCaseGen/external/mom6_bathy/mom6_bathy/grid.py:432, in Grid.subgrid_from_supergrid(cls, path, llc, urc, name)
411 @classmethod
412 def subgrid_from_supergrid(cls, path: str, llc: tuple[float, float], urc: tuple[float, float], name: str) -> "Grid":
413 """Create a Grid instance from a subset of a supergrid file.
414
415 Parameters
(...)
429 The Grid instance created from the supergrid file.
430 """
--> 432 full_grid = cls.from_supergrid(path)
434 assert len(llc) == 2, "llc must be a tuple of two floats"
435 assert len(urc) == 2, "urc must be a tuple of two floats"
File ~/work/CrocoGallery/CrocoGallery/CrocoDash/CrocoDash/visualCaseGen/external/mom6_bathy/mom6_bathy/grid.py:376, in Grid.from_supergrid(cls, path, name)
359 """Create a Grid instance from a supergrid file.
360
361 Parameters
(...)
372 The Grid instance created from the supergrid file.
373 """
375 # read supergrid dataset
--> 376 ds = xr.open_dataset(path)
377 assert (
378 ds.x.units == ds.y.units and "degree" in ds.x.units
379 ), "Only degrees units are supported in supergrid files"
381 # check supergrid
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/api.py:572, in open_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)
560 decoders = _resolve_decoders_kwargs(
561 decode_cf,
562 open_backend_dataset_parameters=backend.open_dataset_parameters,
(...)
568 decode_coords=decode_coords,
569 )
571 overwrite_encoded_chunks = kwargs.pop("overwrite_encoded_chunks", None)
--> 572 backend_ds = backend.open_dataset(
573 filename_or_obj,
574 drop_variables=drop_variables,
575 **decoders,
576 **kwargs,
577 )
578 ds = _dataset_from_backend_dataset(
579 backend_ds,
580 filename_or_obj,
(...)
590 **kwargs,
591 )
592 return ds
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:593, in NetCDF4BackendEntrypoint.open_dataset(self, filename_or_obj, mask_and_scale, decode_times, concat_characters, decode_coords, drop_variables, use_cftime, decode_timedelta, group, mode, format, clobber, diskless, persist, lock, autoclose)
572 def open_dataset( # type: ignore[override] # allow LSP violation, not supporting **kwargs
573 self,
574 filename_or_obj: str | os.PathLike[Any] | BufferedIOBase | AbstractDataStore,
(...)
590 autoclose=False,
591 ) -> Dataset:
592 filename_or_obj = _normalize_path(filename_or_obj)
--> 593 store = NetCDF4DataStore.open(
594 filename_or_obj,
595 mode=mode,
596 format=format,
597 group=group,
598 clobber=clobber,
599 diskless=diskless,
600 persist=persist,
601 lock=lock,
602 autoclose=autoclose,
603 )
605 store_entrypoint = StoreBackendEntrypoint()
606 with close_on_error(store):
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:400, in NetCDF4DataStore.open(cls, filename, mode, format, group, clobber, diskless, persist, lock, lock_maker, autoclose)
394 kwargs = dict(
395 clobber=clobber, diskless=diskless, persist=persist, format=format
396 )
397 manager = CachingFileManager(
398 netCDF4.Dataset, filename, mode=mode, kwargs=kwargs
399 )
--> 400 return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose)
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:347, in NetCDF4DataStore.__init__(self, manager, group, mode, lock, autoclose)
345 self._group = group
346 self._mode = mode
--> 347 self.format = self.ds.data_model
348 self._filename = self.ds.filepath()
349 self.is_remote = is_remote_uri(self._filename)
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:409, in NetCDF4DataStore.ds(self)
407 @property
408 def ds(self):
--> 409 return self._acquire()
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:403, in NetCDF4DataStore._acquire(self, needs_lock)
402 def _acquire(self, needs_lock=True):
--> 403 with self._manager.acquire_context(needs_lock) as root:
404 ds = _nc4_require_group(root, self._group, self._mode)
405 return ds
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/contextlib.py:137, in _GeneratorContextManager.__enter__(self)
135 del self.args, self.kwds, self.func
136 try:
--> 137 return next(self.gen)
138 except StopIteration:
139 raise RuntimeError("generator didn't yield") from None
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/file_manager.py:199, in CachingFileManager.acquire_context(self, needs_lock)
196 @contextlib.contextmanager
197 def acquire_context(self, needs_lock=True):
198 """Context manager for acquiring a file."""
--> 199 file, cached = self._acquire_with_cache_info(needs_lock)
200 try:
201 yield file
File /usr/share/miniconda/envs/CrocoDash/lib/python3.11/site-packages/xarray/backends/file_manager.py:217, in CachingFileManager._acquire_with_cache_info(self, needs_lock)
215 kwargs = kwargs.copy()
216 kwargs["mode"] = self._mode
--> 217 file = self._opener(*self._args, **kwargs)
218 if self._mode == "w":
219 # ensure file doesn't get overridden when opened again
220 self._mode = "a"
File src/netCDF4/_netCDF4.pyx:2469, in netCDF4._netCDF4.Dataset.__init__()
File src/netCDF4/_netCDF4.pyx:2028, in netCDF4._netCDF4._ensure_nc_success()
OSError: [Errno -78] NetCDF: Authorization failure: 's3://crocodile-cesm/CrocoDash/data/grids/ocean_hgrid_trimmed.nc'
Step 1.2: Topography#
from CrocoDash.topo import Topo
topo = Topo(
grid = grid,
min_depth = 9.5,
)
bathymetry_path='s3://crocodile-cesm/CrocoDash/data/gebco/GEBCO_2024.zarr/'
topo.interpolate_from_file(
file_path = bathymetry_path,
longitude_coordinate_name="lon",
latitude_coordinate_name="lat",
vertical_coordinate_name="elevation"
)
topo.depth.plot()
TODO: Implement a subset_global_topo-like method in mom6 and use it for this example instead of interpolation.
Step 1.3: Vertical Grid#
from CrocoDash.vgrid import VGrid
vgrid = VGrid.hyperbolic(
nk = 75,
depth = topo.max_depth,
ratio=20.0
)
SECTION 2: Create the CESM case#
After generating the MOM6 domain, the next step is to create a CESM case using CrocoDash. This process is straightforward and involves instantiating the CrocoDash Case object. The Case object requires the following inputs:
CESM Source Directory: A local path to a compatible CESM source copy.
Case Name: A unique name for the CESM case.
Input Directory: The directory where all necessary input files will be written.
MOM6 Domain Objects: The horizontal grid, topography, and vertical grid created in the previous section.
Project ID: (Optional) A project ID, if required by the machine.
from pathlib import Path
# CESM case (experiment) name
casename = "three-boundary-nwa"
# CESM source root (Update this path accordingly!!!)
cesmroot = "/home/runner/work/CrocoGallery/CrocoGallery/CESM/"
# Place where all your input files go
inputdir = Path.home() / "croc_input" / casename
# CESM case directory
caseroot = Path.home() / "croc_cases" / casename
Step 2.2: Create the Case#
To create the CESM case, instantiate the Case object as shown below. This will automatically set up the CESM case based on the provided inputs: The cesmroot argument specifies the path to your local CESM source directory.
The caseroot argument defines the directory where the case will be created. CrocoDash will handle all necessary namelist modifications and XML changes to align with the MOM6 domain objects generated earlier.
from CrocoDash.case import Case
case = Case(
cesmroot = cesmroot,
caseroot = caseroot,
inputdir = inputdir,
ocn_grid = grid,
ocn_vgrid = vgrid,
ocn_topo = topo,
project = 'NCGD0011',
override = False,
)
Section 3: Prepare ocean forcing data#
We need to cut out our ocean forcing. The package expects an initial condition and one time-dependent segment per non-land boundary. Naming convention is "east_unprocessed" for segments and "ic_unprocessed" for the initial condition.
In this notebook, we are forcing with the Copernicus Marine “Glorys” reanalysis dataset. There’s a function in the CrocoDash package, called configure_forcings, that generates a bash script to download the correct boundary forcing files for your experiment. First, you will need to create an account with Copernicus, and then call copernicusmarine login to set up your login details on your machine. Then you can run the get_glorys_data.sh bash script.
Step 3.1 Configure Initial Conditions and Forcings#
case.configure_forcings(
date_range = ["2020-01-01 00:00:00", "2020-02-01 00:00:00"],
boundaries = ["south", "north","east"],
tidal_constituents = ['M2'],
tpxo_elevation_filepath = "s3://crocodile-cesm/CrocoDash/data/tpxo/h_tpxo9.v1.zarr/",
tpxo_velocity_filepath = "s3://crocodile-cesm/CrocoDash/data/tpxo/u_tpxo9.v1.zarr/"
)
Step 3.2 Run get_glorys_data.sh#
In a terminal session, locate the get_glorys_data.sh script and execute it to download the initial conditions and boundary conditions. Follow the instructions printed by the configure_forcings method above.
TODO: user copernicusmarine python API within CrocoDash, instead of directing users to run it via CLI. Also, on a derecho login node, both CLI and API fails to run due to the computational demand. We also need to address that.
Step 3.3: Process forcing data#
In this final step, we call the process_forcings method of CrocoDash to cut out and interpolate the initial condition as well as all boundaries. CrocoDash also updates MOM6 runtime parameters and CESM xml variables accordingly.
case.process_forcings()
Section 4: Build and run the case#
After completing the previous steps, you are ready to build and run your CESM case. Begin by navigating to the case root directory specified during the case creation. Before proceeding, review the user_nl_mom file located in the case directory. This file contains MOM6 parameter settings that were automatically generated by CrocoDash. Carefully examine these parameters and make any necessary adjustments to fine-tune the model for your specific requirements. While CrocoDash aims to provide a solid starting point, further tuning and adjustments are typically necessary to improve the model for your use case.
Once you have reviewed and modified the parameters as needed, you can prepare, build, and execute the case using the following commands:
./case.setup
./case.build
./case.submit