CrocoDash.extract_forcings package

CrocoDash.extract_forcings package#

Submodules#

CrocoDash.extract_forcings.bgc module#

CrocoDash.extract_forcings.bgc.process_bgc_ic(file_path, output_path)#

Copy BGC initial condition file

Parameters: - file_path: str, path to the original BGC IC file - output_path: str, path to save the processed BGC IC file

Returns: - None

CrocoDash.extract_forcings.bgc.process_bgc_iron_forcing(nx, ny, MARBL_FESEDFLUX_FILE, MARBL_FEVENTFLUX_FILE, MARBL_FESEDFLUXRED_FILE, inputdir)#

Create dummy iron forcing files for MARBL. Parameters: - nx: int, number of grid points in x-direction - ny: int, number of grid points in y-direction - MARBL_FESEDFLUX_FILE: str, filename for sediment flux input - MARBL_FEVENTFLUX_FILE: str, filename for event flux input - inputdir: str, directory to save the generated files Returns: - None

CrocoDash.extract_forcings.bgc.process_river_nutrients(global_river_nutrients_filepath, ocn_grid, mapping_file, river_nutrients_nnsm_filepath, calendar: Calendar = Calendar(cf='noleap', cesm='NO_LEAP', mom6='noleap'))#

CrocoDash.extract_forcings.chlorophyll module#

CrocoDash.extract_forcings.chlorophyll.process_chl(ocn_grid, ocn_topo, inputdir, chl_processed_filepath, output_filepath, calendar: Calendar = Calendar(cf='noleap', cesm='NO_LEAP', mom6='noleap'))#

CrocoDash.extract_forcings.driver module#

CrocoDash Forcing Extraction Driver

Orchestrates the forcing extraction workflow for a CrocoDash case. Reads config.json (written by Case.configure_forcings) and derives all paths from the case state file via case_state.read.

This module exposes run_workflow as the Python API. The CLI entry point is crocodash process (see CrocoDash.cli).

Typical Python usage:

from CrocoDash.extract_forcings.driver import run_workflow

run_workflow(config_path="~/croc_input/mycase/extract_forcings/config.json", bc=True, ic=True)
CrocoDash.extract_forcings.driver.resolve_components(args, config)#

Resolve which components to run based on CLI args and config availability.

CrocoDash.extract_forcings.driver.run_workflow(config_path, ic=False, bc=False, bgcic=False, bgcironforcing=False, tides=False, chl_=False, runoff=False, bgcrivernutrients=False, preview=False)#

Execute the forcing extraction workflow.

Parameters:

  • config_path (str or Path) – Path to the config.json written by Case.configure_forcings.

  • ic (bool) – Run initial conditions.

  • bc (bool) – Run boundary conditions.

  • bgcic (bool) – Run BGC initial conditions.

  • bgcironforcing (bool) – Run BGC iron forcing.

  • tides (bool) – Run tidal forcing.

  • chl (bool) – Run chlorophyll processing.

  • runoff (bool) – Run runoff mapping.

  • bgcrivernutrients (bool) – Run BGC river nutrients (always runs after runoff).

  • preview (bool) – Preview task graph without executing.

CrocoDash.extract_forcings.initial_condition module#

CrocoDash.extract_forcings.initial_condition.final_cleanliness_fill(var, x_dim, y_dim, z_dim=None)#
CrocoDash.extract_forcings.initial_condition.process_initial_condition(product_name: str, function_name: str, product_information: dict, start_date: str | datetime, hgrid_path: str | Path, vgrid_path: str | Path, dataset_varnames: dict, raw_data_dir: str | Path, output_data_dir: str | Path, bathymetry_path: str | Path, preview: bool = False, function_args: dict = None)#

Process the initial condition (t=0) through the data retrieval pipeline.

Parameters:

  • product_name – The name of the data product to retrieve.

  • function_name – The function to call for retrieving data.

  • product_information – Variable name mappings and metadata for the forcing product.

  • start_date – The start date (any pandas-parseable string or datetime).

  • hgrid_path – Path to the hgrid supergrid file.

  • vgrid_path – Path to the vertical grid file.

  • dataset_varnames – Variable name mappings passed to rm6 regridding.

  • raw_data_dir – Directory for raw downloaded data.

  • output_data_dir – Directory for final MOM6-ready output files.

  • bathymetry_path – Path to the bathymetry file.

  • preview – Return metadata dict without executing, default False.

  • function_args – Overrides for the access function’s non-required arguments (e.g. member), as written to config.json by configure_forcings()’s function_overrides.

CrocoDash.extract_forcings.obc module#

OBC (Open Boundary Condition) forcing extraction for CrocoDash.

Three-phase pipeline per boundary:

  1. GET — download raw data, chunked by get_step (default: full range in

    one request). Chunk size is driven by data-provider constraints (API limits, download size). Each chunk is written as {boundary}_unprocessed.{start}_{end}.nc.

  2. REGRID — validate raw coverage from filenames, then open all raw files

    lazily and regrid in regrid_step-sized slices. Chunk size is driven by memory and xESMF performance. GET and REGRID chunks are fully independent.

  3. MERGE — concatenate regridded chunks into forcing_obc_segment_NNN.nc.

Each phase is idempotent: existing output files are detected and skipped, so a failed run can be safely re-started.

CrocoDash.extract_forcings.obc.process_obc_conditions(start_date, end_date, boundary_number_conversion: dict, product_name: str, function_name: str, product_info: dict, hgrid_path, raw_dataset_path, regridded_dataset_path, output_path, get_step_days=None, regrid_step_days: int = 30, function_args: dict = None, bathymetry_path=None, preview: bool = False)#

Process boundary conditions through the GET → REGRID → MERGE pipeline.

Each phase is idempotent. Re-running after a partial failure resumes from the last completed file.

GET and REGRID chunk sizes are independent. GET defaults to the full date range in one request; REGRID defaults to 30-day slices for memory efficiency.

Parameters:

  • start_date – Forcing start date (datetime or any pandas-parseable string).

  • end_date – Forcing end date (datetime or any pandas-parseable string).

  • boundary_number_conversion – Boundary name -> MOM6 segment number.

  • product_name – Forcing data product name.

  • function_name – Download function name for the product.

  • product_info – Product variable-name metadata (a.k.a. dataset_varnames).

  • hgrid_path – Path to the hgrid supergrid file.

  • raw_dataset_path – Directory for raw downloaded data.

  • regridded_dataset_path – Directory for per-chunk regridded data.

  • output_path – Directory for final, merged MOM6-ready output files.

  • get_step_days – GET chunk size in days; None = full range in one request.

  • regrid_step_days – REGRID chunk size in days.

  • function_args – Overrides for the access function’s non-required arguments (e.g. member), as resolved by configure_forcings()’s function_overrides.

  • bathymetry_path – Optional path to the case’s bathymetry file. When given, download bounding boxes are computed from the bathymetry ocean tmask (tighter than the full supergrid edge extent). When omitted, falls back to the full supergrid bounding box per boundary.

  • preview – If True, return a dict of expected date pairs without executing any downloads or regridding.

CrocoDash.extract_forcings.runoff module#

CrocoDash.extract_forcings.runoff.generate_rof_ocn_map(rof_grid_name, rof_esmf_mesh_filepath, ocn_mesh_filepath, inputdir, grid_name, rmax, fold)#

Generate runoff to ocean mapping files if runoff is active in the compset.

CrocoDash.extract_forcings.tides module#

CrocoDash.extract_forcings.tides.process_tides(ocn_topo, inputdir, supergrid_path, vgrid_path, tidal_constituents, boundaries, tpxo_elevation_filepath, tpxo_velocity_filepath)#

CrocoDash.extract_forcings.utils module#

CrocoDash.extract_forcings.utils.build_forcing_request(product_info: dict, function_args: dict = None) → tuple[list, dict]#

Build the (variables, extra_args) an access function needs from a forcing product_info dict.

function_args: user overrides (or access-function defaults) for the access function’s non-required arguments, as written to config.json’s forcing.function_args by configure_forcings()’s function_overrides. Merged into extra_args last so they take precedence over product_info-derived keys.

CrocoDash.extract_forcings.utils.check_date_continuity(boundary_file_list: dict)#

Check for overlaps or missing dates between consecutive files.

CrocoDash.extract_forcings.utils.fetch_raw_chunk(data_access_fn, dates: list, latlon: dict, output_folder: str | Path, output_filename: str, variables: list, extra_args: dict, name=None) → Path#

Download one raw data chunk, skipping if a valid output file already exists.

Shared by obc.py and initial_condition.py — both fetch a chunk of raw data for a given date range and bounding box, and both need to be idempotent across re-runs.

CrocoDash.extract_forcings.utils.get_data_access_function(product_name: str, function_name: str)#

Load the product registry and return the raw access function for (product_name, function_name).

CrocoDash.extract_forcings.utils.is_valid_netcdf(path: Path) → bool#

Check a file’s magic bytes match a known NetCDF format (HDF5, classic, or 64-bit offset).

CrocoDash.extract_forcings.utils.parse_dataset_folder(folder: str | Path, input_dataset_regex: str, date_format: str)#

Parse a folder to find and extract dataset file information based on a regex pattern.

Parameters:

  • folder (str or Path) – Path to the folder containing the dataset files.

  • input_dataset_regex (str) – Regular expression pattern to match dataset filenames. Example: “(north|east|south|west)_unprocessed.(d{8})_(d{8}).nc”

  • date_format (str) – Date format string used to parse dates in filenames (e.g., “%Y%m%d”).

Returns:

Dictionary mapping boundaries to a list of tuples with: - Start date (datetime) - End date (datetime) - Full file path (Path)

Example: {

”north”: [(datetime(2000, 1, 1), datetime(2000, 1, 2), Path(“/path/to/north_20000101_20000102.nc”))], “east”: [(datetime(2000, 1, 3), datetime(2000, 1, 4), Path(“/path/to/east_20000103_20000104.nc”))]

}

Return type:

dict

Module contents#

Contents