ewatercycle.config

Config

Configuration of eWaterCycle is done via the Configuration object. The global configuration can be imported from the eWaterCycle module as CFG:

>>> from ewatercycle import CFG
>>> CFG
Configuration(
    grdc_location=PosixPath('.'),
    container_engine='docker',
    apptainer_dir=PosixPath('.'),
    singularity_dir=None,
    output_dir=PosixPath('.'),
    parameterset_dir=PosixPath('.'),
    parameter_sets={},
    ewatercycle_config=None
)

By default all values have usable values.

CFG is a Pydantic model. This means that values can be updated like this:

>>> CFG.output_dir = '~/output'
>>> CFG.output_dir
PosixPath('/home/user/output')

Notice that CFG automatically converts the path to an instance of pathlib.Path and expands the home directory. All values entered into the config are validated to prevent mistakes, for example, it will warn you if you make a typo in the key:

>>> CFG.output_directory = '/output'
ValidationError: 1 validation error for Configuration
output_directory
    extra fields not permitted (type=value_error.extra)

Or, if the value entered cannot be converted to the expected type:

>>> CFG.output_dir = 123
ValidationError: 1 validation error for Configuration
output_dir
    value is not a valid path (type=type_error.path)

By default, the config is loaded from the default location (i.e. ~/.config/ewatercycle/ewatercycle.yaml). If it does not exist, it falls back to the default values. to load a different file:

>>> CFG.load_from_file('~/my-config.yml')

Or to reload the current config:

>>> CFG.reload()
ewatercycle.config.CFG

eWaterCycle configuration object.

The configuration is loaded from:

  1. $XDG_CONFIG_HOME/ewatercycle/ewatercycle.yaml

  2. ~/.config/ewatercycle/ewatercycle.yaml

  3. /etc/ewatercycle.yaml

  4. Fall back to empty configuration

The ewatercycle.yaml is formatted in YAML and could for example look like:

grdc_location: /data/grdc
container_engine: apptainer
apptainer_dir: /data/apptainer-images
output_dir: /scratch
# Filled apptainer_dir with
# cd /data/apptainer-images
# apptainer pull docker://ewatercycle/wflow-grpc4bmi:2020.1.1

Package Contents

class ewatercycle.config.ParameterSet

Bases: pydantic.BaseModel

Container object for parameter set options.

Is directory containing data that does not change over time. Should be passed to a models constructor like AbstractModel.

Example

>>> from ewatercycle.base import ParameterSet
>>> parameter_set = ParameterSet(name='test', directory='test_dir', config='test_dir/config.yaml')
>>> from ewatercycle.models import Wflow
>>> model = Wflow(parameter_set=parameter_set)
name: str = ''

Name of parameter set

directory: pathlib.Path

Location on disk where files of parameter set are stored.

If Path is relative then relative to CFG.parameterset_dir.

config: pathlib.Path

Model configuration file which uses files from directory.

If Path is relative then relative to directory.

doi: str = 'N/A'

Persistent identifier of parameter set.

For a example a DOI for a Zenodo record.

target_model: str = 'generic'

Name of model that parameter set can work with.

supported_model_versions: Set[str]

Set of model versions that are compatible with this parameter set.

If not set then parameter set compability check silently passes.

downloader: GitHubDownloader | ZenodoDownloader | ArchiveDownloader | None

Method to download parameter set from somewhere.

model_config
download(download_dir: pathlib.Path, force: bool = False) → None

Download parameter set to directory.

Parameters:

  • download_dir – Directory where parameter set should be downloaded to.

  • force – If True then download even if directory already exists.

Raises:

ValueError – If no downloader is defined.

classmethod from_github(org: str, repo: str, branch: str, subfolder: str | None = None, **kwargs)

Create a parameter set from a GitHub repository.

Parameters:

  • org – GitHub organization (e.g., ‘UU-Hydro’)

  • repo – Repository name (e.g, ‘PCR-GLOBWB_input_example’)

  • branch – Branch name (e.g., ‘master’)

  • subfolder – Subfolder within the github repo to extract. E.g. ‘pcrglobwb_rhinemeuse_30min’. If not given then downloads the entire repository.

  • **kwargs – See ParameterSet for other arguments.

The example Github arguments would download all files from https://github.com/UU-Hydro/PCR-GLOBWB_input_example/tree/master/RhineMeuse30min

classmethod from_zenodo(doi: str, **kwargs) → ParameterSet

Download a parameter set from Zenodo.

Parameters:

  • doi – DOI of Zenodo record. E.g. ‘10.5281/zenodo.1045339’.

  • **kwargs – See ParameterSet for other arguments.

classmethod from_archive_url(url: str, **kwargs) → ParameterSet

Download a parameter set from an archive file on the Internet.

Parameters:

  • url – Link to archive file.

  • **kwargs – See ParameterSet for other arguments.

Example

>>> from ewatercycle.base import ParameterSet
>>> url = ''
>>> parameter_set = ParameterSet.from_archive_url(url)
make_absolute(parameterset_dir: pathlib.Path) → ParameterSet

Make self.directory and self.config absolute paths. :param parameterset_dir: Directory to which relative paths should be made absolute.

Returns:

self

ewatercycle.config.to_absolute_path(input_path: str | pathlib.Path, parent: pathlib.Path | None = None, must_exist: bool = False, must_be_in_parent=True) → pathlib.Path

Parse input string as pathlib.Path object.

Parameters:

  • input_path – Input string path that can be a relative or absolute path.

  • parent – Optional parent path of the input path

  • must_exist – Optional argument to check if the input path exists.

  • must_be_in_parent – Optional argument to check if the input path is subpath of parent path

Returns:

The input path that is an absolute path and a pathlib.Path object.

ewatercycle.config.logger
ewatercycle.config.ContainerEngine
ewatercycle.config.ExpandedDirectoryPath
ewatercycle.config.ExpandedFilePath
class ewatercycle.config.Configuration

Bases: pydantic.BaseModel

Configuration object.

Do not instantiate this class directly, but use ewatercycle.CFG instead.

grdc_location: ExpandedDirectoryPath

Where can GRDC observation files (<station identifier>_Q_Day.Cmd.txt) be found.

container_engine: ContainerEngine = 'docker'

Which container engine is used to run the hydrological models.

apptainer_dir: ExpandedDirectoryPath

Where the apptainer images files (.sif) be found.

singularity_dir: pydantic.DirectoryPath | None

Where the singularity images files (.sif) be found.

DEPRECATED, use apptainer_dir.

output_dir: ExpandedDirectoryPath

Directory in which output of model runs is stored.

Each model run will generate a sub directory inside output_dir

parameterset_dir: ExpandedDirectoryPath

Root directory for all parameter sets.

parameter_sets: Dict[str, ewatercycle.base.parameter_set.ParameterSet]

Dictionary of parameter sets.

Data source for ewatercycle.parameter_sets.available_parameter_sets() and ewatercycle.parameter_sets.get_parameter_set() methods.

ewatercycle_config: ExpandedFilePath | None

Where is the configuration saved or loaded from.

If None then the configuration was not loaded from a file.

model_config
classmethod singularity_dir_is_deprecated(data: Any) → Any
load_from_file(filename: os.PathLike[str] | str) → None

Load user configuration from the given file.

The config is cleared and updated in-place.

reload() → None

Reload the config file.

reset() → None

Reset to empty configuration.

dump_to_yaml() → str

Dumps YAML formatted string of Config object

save_to_file(config_file: os.PathLike[str] | str | None = None) → None

Write conf object to a file.

Parameters:

config_file – File to write configuration object to. If not given then will try to use self.ewatercycle_config location and if self.ewatercycle_config is not set then will use the location in users home directory.

overwrite(other: Configuration)

Overwrite own fields by the ones of the other configuration object.

Parameters:

other – The other configuration object.

ewatercycle.config.USER_HOME_CONFIG
ewatercycle.config.SYSTEM_CONFIG
ewatercycle.config.USER_CONFIG
ewatercycle.config.CFG
ewatercycle.config.CFG