CrocoDash.raw_data_access package

CrocoDash.raw_data_access package#

Subpackages#

CrocoDash.raw_data_access.datasets package

Submodules#

CrocoDash.raw_data_access.base module#

Raw Data Access Module Requirements: 1. Must enforce a certain amount of metadata for each product (things like z coordinate name, u velocity name, etc) (from the parent class) 2. Must enforce specific keyword args in each access function (from the parent class) 3. There must be some sort of registry that can list/query all products & access methods 4. As minimal overhead as possible (for me, that meant no initialization!) 5. Should be easy to validate the function (easy to test if they work and the metadata) 6. Should be easy to validate the metadata 7. Need to be able to print out the available products, functions, and their respective descriptions.

How it was solved: 1. There is a set of “abstract” classes that specify an array of “required_metadata”. There is no type enforcement currently. This is checked in the _init_subclass hook. 2. There is a set of “abstract” classes that specify an array of “required_args”. There is no type enforcement. This is checked in the _init_subclass hook. 3. There is a class called ProductRegistry in the registry.py file that does this 4. The classes are all static. 5. There is a validate method in the BaseProduct class that takes in additional default args from child classes, and the _init_subclass hook validates the args. 6. The metadata is validated in the _init_subclass hook 7. The ProductRegistry holds all of that.

class CrocoDash.raw_data_access.base.BaseProduct#

Bases: object

Base class for all raw data products. It enforces the metadata on the product as well as the function args.

required_args = ['output_folder', 'output_filename']#

required_metadata = ['product_name', 'description', 'link']#

classmethod validate_call(method_name, **kwargs)#: Validate that a call to an access method has correct arguments.

classmethod validate_method(method_name, **kwargs)#: Default validation just makes a toy call with a temporary directory.

classmethod write_metadata(file_path: str = None) → dict#: Return a dict of the class metadata fields and their values, writes a file if a filepath is specified.

class CrocoDash.raw_data_access.base.DatedBaseProduct#

Bases: BaseProduct

Specific enforcement needs for Dated Products

required_args = ['output_folder', 'output_filename', 'dates']#

classmethod validate_method(method_name, **kwargs)#: Default validation just makes a toy call with a temporary directory.

class CrocoDash.raw_data_access.base.ForcingProduct#

Bases: DatedBaseProduct

Specific enforcement needs for Forcing Products

required_args = ['output_folder', 'output_filename', 'dates', 'variables', 'lon_max', 'lat_max', 'lon_min', 'lat_min']#

required_metadata = ['product_name', 'description', 'link', 'time_var_name', 'u_x_coord', 'u_y_coord', 'v_x_coord', 'v_y_coord', 'tracer_x_coord', 'tracer_y_coord', 'depth_coord', 'u_var_name', 'v_var_name', 'eta_var_name', 'tracer_var_names', 'boundary_fill_method', 'time_units']#

classmethod validate_method(method_name, **kwargs)#: Default validation just makes a toy call with a temporary directory.

classmethod write_metadata(file_path: str | None = None, include_marbl_tracers=False) → dict#: Return a dict of the class metadata fields and their values, writes a file if a filepath is specified.

CrocoDash.raw_data_access.base.accessmethod(func=None, *, description=None, type=None)#

CrocoDash.raw_data_access.registry module#

class CrocoDash.raw_data_access.registry.ProductRegistry#

Bases: object

Static registry that tracks all products and provides driver-like introspection.

classmethod call(product_name, method_name, **kwargs)#

classmethod get_access_function(product_name, method_name)#: Return the raw function (unbound), even if staticmethod.

classmethod get_product(name)#

classmethod list_access_methods(name)#

classmethod list_products()#

classmethod load()#

loaded = False#

classmethod product_exists(name)#

classmethod product_is_of_type(name, the_class)#

products = {}#

classmethod register(product_cls)#: Register a product class.

classmethod validate_function(product_name, method_name)#

CrocoDash.raw_data_access.utils module#

CrocoDash.raw_data_access.utils.fill_template(template_path, output_path, **kwargs)#

Reads a template file, fills it with the provided arguments, and writes the filled content to a new file.

Parameters:

template_path (str) – Path to the template file.
output_path (str) – Path to save the filled template.
**kwargs – Key-value pairs for substitution in the template.

CrocoDash.raw_data_access.utils.load_config(config_path: str = 'config.json') → dict#

Load a JSON config file.

Parameters:: config_path (str, optional) – Path to the JSON config file. Default is “config.json”.
Returns:: The loaded configuration as a dictionary.
Return type:: dict

CrocoDash.raw_data_access.utils.setup_logger(name)#

This function sets up a logger format for the package. It attaches logger output to stdout (if a handler doesn’t already exist) and formats it in a pretty way!

Parameters:: name (str) – The name of the logger.
Returns:: The logger
Return type:: logging.Logger