Feature modelblocks

.copier-answers.yml

@@ -1,13 +1,13 @@
 # Changes here will be overwritten by Copier
-# DO NOT MANUALLY MODIFY THIS FILE
+# !!!!! DO NOT MANUALLY MODIFY THIS FILE !!!!!
 _commit: latest
 _src_path: https://github.com/calliope-project/data-module-template.git
 author_email: i.ruizmanuel@tudelft.nl
 author_family_name: Ruiz Manuel
 author_given_name: Ivan
-github_org: calliope-project
+github_org: modelblocks-org
 license: Apache-2.0
 module_description: A module to calculate hydropower inflow timeseries for facilities
-    around the globe. Based on Euro-Calliope methods.
-module_long_name: clio - Hydropower module
+    around the globe, based on Euro-Calliope methods.
+module_long_name: Modelblocks - Hydropower module
 module_short_name: module_hydropower

.github/ISSUE_TEMPLATE/bug_report.md

@@ -24,7 +24,7 @@ A clear and concise description of what you expected to happen.
 If applicable, add screenshots to help explain your problem.
 
 **Desktop (please complete the following information):**
- - OS: [e.g. Linux Fedora 41, Windows 10...]
+ - OS: [e.g. Linux Fedora 43, Windows 11...]
  - Version [e.g. v0.1.1]
 
 **Additional context**

.github/ISSUE_TEMPLATE/config.yaml

@@ -0,0 +1,5 @@
+blank_issues_enabled: false
+contact_links:
+  - name: General information
+    url: https://www.modelblocks.org/
+    about: Please consult our website for general information on the Modelblocks methodology.

.github/pull_request_template.md

@@ -8,6 +8,11 @@ Fixes #
 
 ## Reviewer checklist
 
-* [ ] `INTERFACE.yaml` is up-to-date with all relevant user resources and results.
-* [ ] The integration example is up-to-date with a minimal use-case of the module.
-* [ ] Module documentation is up-to-date.
+* [ ] There are no `pip` dependencies in the module's environment files (`workflow/envs/`).
+* [ ] All rules use `pathvars` (e.g., `<results>`) in their inputs and outputs.
+* [ ] The integration test-suite is successful, including:
+    * [ ] `pre-commit.ci` tests pass.
+    * [ ] tests pass for all relevant OS configurations (linux, osx, windows).
+* [ ] Module documentation is up-to-date, including:
+    * [ ] `INTERFACE.yaml` mentions all relevant `pathvars` and `wildcards`.
+    * [ ] `README.md` describes how to use the module and has the necessary citations.

.github/workflows/check-version.yml

@@ -6,4 +6,4 @@ on:
 
 jobs:
   copier-update:
-    uses: calliope-project/data-module-template/.github/workflows/template-check-version.yml@latest
+    uses: modelblocks-org/data-module-template/.github/workflows/template-check-version.yml@latest

.github/workflows/pr-ci.yml

@@ -1,11 +1,43 @@
-name: CI
+# !!!!! DO NOT MODIFY UNLESS ABSOLUTELY NECESSARY !!!!!
+# This workflow helps standardise integration tests across data modules
+name: Pull Request CI tests for Modelblocks data modules
 on:
   pull_request:
     branches:
       - "main"
-
 jobs:
-  ci-tests:
-    uses: calliope-project/data-module-template/.github/workflows/template-pr-ci.yml@feature-ci-cdsapi
-    secrets:
+  build:
+    name: Build
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
+        python-version: ["3.12"]
+    env:
       CDSAPI_KEY: ${{ secrets.CDSAPI_KEY }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: Setup pixi
+        uses: prefix-dev/setup-pixi@v0.8.3
+      - name: Run integration tests
+        id: tests
+        run: pixi run test-integration
+        continue-on-error: true
+      - name: Run snakemake linting
+        id: linting
+        run: |
+          pixi run snakemake --lint --snakefile workflow/Snakefile
+          pixi run snakemake --lint --snakefile tests/integration/Snakefile
+        continue-on-error: true
+      - name: Save integration logs
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: integration-test-logs-${{ matrix.os }}
+          path: tests/integration/results/integration_test/logs
+          if-no-files-found: ignore
+          retention-days: 30
+      - name: Fail if integration or linting failed
+        if: ${{ steps.tests.outcome == 'failure' || steps.linting.outcome == 'failure' }}
+        run: exit 1

.github/workflows/release.yml

@@ -6,4 +6,6 @@ on:
 
 jobs:
   release-workflow:
-    uses: calliope-project/data-module-template/.github/workflows/template-release.yml@latest
+    permissions:
+      contents: write
+    uses: modelblocks-org/data-module-template/.github/workflows/template-release.yml@latest

.pre-commit-config.yaml

@@ -12,6 +12,7 @@ repos:
       # Repo quality
       - id: check-added-large-files
         args: [--enforce-all --maxkb=600]
+        exclude: ^pixi\.lock$
       - id: forbid-submodules
       - id: check-case-conflict
       - id: check-illegal-windows-names
@@ -32,7 +33,7 @@ repos:
 
   # Snakemake file formatting
   - repo: https://github.com/snakemake/snakefmt
-    rev: v0.11.0
+    rev: v0.11.4
     hooks:
       - id: snakefmt
 
@@ -44,4 +45,4 @@ repos:
         files: .*\.(py|smk|md)$|^Snakefile$
 
 ci:  # https://pre-commit.ci/
-  autoupdate_schedule: monthly
+  autoupdate_schedule: quarterly

AUTHORS

@@ -2,6 +2,6 @@ This is the list of contributors for copyright purposes.
 
 This does not necessarily list everyone who has contributed to this software's
 code or documentation. For a full contributor list, see:
-<https://github.com/calliope-project/module_hydropower/graphs/contributors>
+<https://github.com/modelblocks-org/module_hydropower/graphs/contributors>
 
 Ivan Ruiz Manuel, <i.ruizmanuel@tudelft.nl>

CITATION.cff

@@ -3,8 +3,8 @@
 # https://citation-file-format.github.io/
 cff-version: 1.2.0
 message: If you use this software or data produced by it, please cite it using the metadata from this file.
-title: "clio - module_hydropower: clio - Hydropower module"
-repository: "https://github.com/calliope-project/module_hydropower"
+title: "Modelblocks - module_hydropower: Hydropower module"
+repository: "https://github.com/modelblocks-org/module_hydropower"
 license: Apache-2.0
 authors:
   - given-names: Ivan

INTERFACE.yaml

@@ -1,17 +1,39 @@
 # Module Input-Output structure for automated doc. generation
-resources:
-  user:
-    "{shapes}/powerplants.parquet": Powerplant statistics.
-    "{shapes}/shapes.parquet": Resolution shapes, used for capacity aggregation.
-results:
-  "{shapes}/statistics/generation.parquet": >
-    National hydropower generation statistics for the requested shapes.
-  "{shapes}/disaggregated/inflow_mwh.parquet": >
-    Inflow time series for each powerplant in MWh.
-    Only includes powerplants that fall within the provided shapes after buffering.
-  "{shapes}/aggregated/reservoir_cf.parquet": >
-    Capacity Factors of hydropower reservoirs aggregated to shapes.
-  "{shapes}/aggregated/run_of_river_cf.parquet": >
-    Capacity Factors of run of river plants aggregated to shapes.
+pathvars:
+  snakemake_defaults:
+    logs:
+      default: "<logs>"
+      description: location of rule log files.
+    resources:
+      default: "<resources>"
+      description: "location of module resource files."
+    results:
+      default: "<results>"
+      description: "location of module results."
+  user_resources:
+    shapes:
+      default: "<resources>/user/{shapes}/shapes.parquet"
+      description: |
+        Region-specific polygons to process during aggregation.
+        See the schemas in `workflow/scripts/_schemas.py` for more information.
+    powerplants:
+      default: "<resources>/user/{shapes}/powerplants.parquet"
+      description: |
+        Powerplant statistics.
+        See the schemas in `workflow/scripts/_schemas.py` for more information.
+  results:
+    statistics:
+      default: "<results>/{shapes}/statistics/generation.parquet"
+      description: National hydropower generation statistics for the requested shapes.
+    disaggregated_inflow:
+      default: "<results>/{shapes}/disaggregated/inflow_mwh.parquet"
+      description: >
+        Inflow time series for each powerplant in MWh.
+        Only includes powerplants that fall within the provided shapes and buffer distance.
+    aggregated_inflow_pu:
+      default: "<results>/{shapes}/aggregated/{plant_type}_inflow_pu.parquet"
+      description: >
+        Inflow per-unit (PU) of capacity timeseries of a hydropower plant type, per shapes.
 wildcards:
   shapes: name of the provided shapes case.
+  plant_type: Either 'run_of_river' or 'reservoir'.

LICENSE

@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright 2025 AUTHORS
+   Copyright 2026 AUTHORS
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -1,21 +1,79 @@
-# clio - Hydropower module
+# Modelblocks - Hydropower module
 
-A module to calculate hydropower inflow timeseries for facilities around the globe. Based on Euro-Calliope methods.
+A module to calculate hydropower inflow timeseries for facilities around the globe, based on Euro-Calliope methods.
 
-A modular `snakemake` workflow built for [`clio`](https://clio.readthedocs.io/) data modules.
+<!-- Place an attractive image of module outputs here -->
+<p align="center">
+  <img src="./figures/reservoir_inflow_pu.png" width="80%">
+</p>
 
-## Using this module
+## About
+<!-- Please do not modify this templated section -->
 
-This module can be imported directly into any `snakemake` workflow.
-Please consult the integration example in `tests/integration/Snakefile` for more information.
+This is a modular `snakemake` workflow created as part of the [Modelblocks project](https://www.modelblocks.org/). It can be imported directly into any `snakemake` workflow.
+
+For more information, please consult the Modelblocks [documentation](https://modelblocks.readthedocs.io/en/latest/),
+the [integration example](./tests/integration/Snakefile),
+and the `snakemake` [documentation](https://snakemake.readthedocs.io/en/stable/snakefiles/modularization.html).
+
+## Overview
+<!-- Please describe the processing stages of this module here -->
+
+Data processing steps:
+
+<p align="center">
+  <img src="./figures/rulegraph.png" width="60%">
+</p>
+
+1. A global dataset of hydro basins is created for a requested Pfafstetter level using data from the [HydroBASINS](https://www.hydrosheds.org/products/hydrobasins) dataset.
+2. Individual powerplant locations (provided by the user) are adjusted using a buffer distance to ensure their location is within the nearest basin.
+
+<p align="center">
+  <img src="./figures/adjusted_powerplants.png" width="60%">
+</p>
+
+3. User provided shapes, powerplants and configuration are used to construct a data request to the [Copernicus Data Store](https://cds.climate.copernicus.eu/) using the [`atlite` library](https://github.com/PyPSA/atlite).
+4. `atlite` is used to construct inflow timeseries per powerplant.
+5. Inflow timeseries are combined and aggregated to the requested resolution, using national-level statistics from the [EIA](https://www.eia.gov/international/) as a guiding normalisation heuristic for total generation.
+6. Per-unit inflow timeseries ($PU_{t,r}$) are produced for each region with available capacity.
+The relation is $InflowMWh_{t,r} = PU_{t,r} \cdot Cap_{r}$, where $Cap_r$ is the capacity per region.
+    - For run of river powerplants, the timeseries are capped so they may not exceed available capacity.
+    - For reservoirs, the timeseries are capped so inflow cannot exceed 10 times the available capacity.
+
+<p align="center">
+  <img src="./figures/reservoir_inflow_pu.png" width="60%">
+</p>
+
+> [!CAUTION]
+> Please be aware of the following limitations.
+>
+> The module assumes the shapes and powerplants provided by the user correspond to national totals, meaning that (for now) the module **will not adequately process subnational cases unless they are a subset of the total national scope**.
+>
+> The module adjusts inflow timeseries using national generation totals as a way to estimate the per-region output in each year.
+> This approach is a **heuristic**.
+
+## Configuration
+<!-- Please describe how to configure this module below -->
+
+Please consult the configuration [README](./config/README.md) and the [configuration example](./config/config.yaml) for a general overview on the configuration options of this module.
+
+## Input / output structure
+<!-- Please describe input / output file placement below -->
+
+As input, you need a file with the polygons to aggregate into (the 'shapes'), and a file specifying national hydropower plants (either RoR or reservoir). These files should follow the `pandera` schemas specified in [_schemas.py](./workflow/scripts/_schemas.py), and can be created using other Modelblocks modules.
+
+Outputs include aggregated timeseries per hydropower plant type, disaggregated inflow timeseries (per point powerplant), and national generation statistics.
+
+Please consult the [interface file](./INTERFACE.yaml) for more information.
 
 ## Development
+<!-- Please do not modify this templated section -->
 
 We use [`pixi`](https://pixi.sh/) as our package manager for development.
-Once installed, run the following to clone this repo and install all dependencies.
+Once installed, run the following to clone this repository and install all dependencies.
 
 ```shell
-git clone git@github.com:calliope-project/module_hydropower.git
+git clone git@github.com:modelblocks-org/module_hydropower.git
 cd module_hydropower
 pixi install --all
 ```
@@ -26,16 +84,30 @@ For testing, simply run:
 pixi run test-integration
 ```
 
-To view the documentation locally, use:
-
-```shell
-pixi run serve-docs
-```
-
 To test a minimal example of a workflow using this module:
 
 ```shell
 pixi shell    # activate this project's environment
 cd tests/integration/  # navigate to the integration example
 snakemake --use-conda --cores 2  # run the workflow!
 ```
+
+## References
+<!-- Please provide thorough referencing below -->
+
+This module is based on the following research and datasets:
+
+* **Basins: HydroSHEDS.**
+Lehner, B., Grill G. (2013). Global river hydrography and network routing: baseline data and new approaches to study the world’s large river systems. Hydrological Processes, 27(15): 2171–2186. <https://doi.org/10.1002/hyp.9740>
+    - Please consult the [declaration statement](./workflow/internal/HydroSHEDS.txt) in this repository for further details.
+* **Inflow timeseries dataset.**
+Hersbach H, Bell B, Berrisford P, et al. The ERA5 global reanalysis. Q J R Meteorol Soc. 2020;146:1999–2049. <https://doi.org/10.1002/qj.3803>
+* **Inflow timeseries processing.**
+Hofmann et al., (2021). atlite: A Lightweight Python Package for Calculating Renewable Power Potentials and Time Series. Journal of Open Source Software, 6(62), 3294, <https://doi.org/10.21105/joss.03294>
+* **Reference source code for national adjustment heuristic.**
+Tröndle, T., & Pickering, B. (2021). Euro-Calliope (Version 1.2.0.dev) [Computer software]. <https://doi.org/10.5281/zenodo.3949793>
+    - MIT licensed. Please consult our source code for details.
+* **National hydropower generation dataset.** U.S. Energy Information Administration (Oct 2008). <https://www.eia.gov/international/overview/world>
+
+Additionally, this module relies on the following for testing and stable integration:
+* **Data stability aid.** Ruiz Manuel, I., & Pfenninger, S. (2026). Modelblocks - module hydropower (v0.1.0) [Data set]. Zenodo. <https://doi.org/10.5281/zenodo.19401947>

config/README.md

@@ -1,9 +1,8 @@
-# Configuration
+We recommend consulting the following before using this module:
+- `config/config.yaml`: a generic example configuration of this module.
+- `workflow/internal/config.schema.yaml`: a schematic overview of all the configuration options of this module.
+- `INTERFACE.yaml`: lists module input and output files, and their default locations.
+- `tests/integration/Snakefile`: an example of how to call this module from another workflow.
 
-This workflow is part of the [clio project](https://clio.readthedocs.io/en/latest/).
-Please consult our documentation for more details.
-Other useful resources are:
-
-- `INTERFACE.yaml`: user input files (placed in `resources/user`) and module output files (placed in `results`).
-- `workflow/internal/config.schema.yaml`: general configuration options.
-- `tests/integration/`: a simple example of how to use this module.
+This data module is part of the [Modelblocks](https://www.modelblocks.org/) project.
+Please consult the [Modelblocks documentation](https://modelblocks.readthedocs.io/) for more details.

config/config.yaml

@@ -14,4 +14,4 @@ powerplants:
 crs:
   projected: "epsg:3857"
   geographic: "epsg:4326"
-smoothing_hours: "168h"
+smoothing_hours: "72h"

figures/.gitkeep

@@ -0,0 +1 @@
+# For module documentation figures