Skip to content

12 generalize downloader #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
liellnima wants to merge 38 commits into
base: main
Choose a base branch
from
Open

12 generalize downloader #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
9e92fbc
remove model source center and reformat
liellnima Oct 22, 2024
7e08575
move selected scenario mip files to docs
liellnima Oct 26, 2024
68dabf5
update download configs for project, and ensemble members
liellnima Oct 26, 2024
114eb10
remove unused esm_constants
liellnima Oct 26, 2024
47f0c78
add new constant files for each esgf project type
liellnima Oct 26, 2024
aa89ff6
remove get_selected_scenario as it is too restricting
liellnima Oct 26, 2024
a61dc2a
remove restricting funcs, extend to broader model set, extend to broa…
liellnima Oct 26, 2024
b868033
move constants into constant classes, and collect them in a dict in e…
liellnima Nov 19, 2024
bb7b8f1
update configs: move project id to the top
liellnima Nov 19, 2024
5a0c38f
update download_from_config func with new constant and config handlin…
liellnima Nov 19, 2024
ad5e0b0
Add base structure for abstract downloader and implementations
f-PLT Jan 10, 2025
aa0e451
Refactor ESGF constants and project constants
f-PLT Jan 28, 2025
6a76fa9
Add first base structure of Config classes
f-PLT Jan 28, 2025
23b0bea
Integrate config class for Input4mips
f-PLT Jan 28, 2025
770c003
Implement config classes
f-PLT Feb 26, 2025
0ea3aae
Update tests
f-PLT Feb 26, 2025
9df2456
Refactor CMIP6Downloader for multiple models
f-PLT Feb 26, 2025
df32336
Cleanup of downloader.py file
f-PLT Feb 26, 2025
1f2ff66
Cleanup of downloader.py file
f-PLT Feb 26, 2025
7789c1b
Update all download config files
f-PLT Feb 27, 2025
ecf8a41
Add download example
f-PLT Feb 27, 2025
260952d
Update download_from_config_file() to use existing functions for each…
f-PLT Feb 27, 2025
1017f14
Fix Pylint errors
f-PLT Feb 27, 2025
445115f
fix typo
liellnima Feb 27, 2025
51614eb
update minimal usecase config and add ocean configs for future
liellnima Feb 27, 2025
30e3969
add ocean constants for future use cases, can be ignored rn
liellnima Feb 27, 2025
2c341dc
Update with new QA tools and new Makefile version
f-PLT May 23, 2025
73b56a2
Ruff fix lint + formatting
f-PLT May 23, 2025
4f0283b
Update and fix failing test
f-PLT May 23, 2025
0ceafa0
Refactor input4mips constants for safety
f-PLT May 23, 2025
3ed950c
Handle pylint warnings
f-PLT May 23, 2025
99561a3
Update github actions
f-PLT May 24, 2025
bf53e46
Formatting for pyproject.toml
f-PLT May 26, 2025
665d77b
Refactor downloader constants
f-PLT May 28, 2025
6717393
Refactor downloader_config from Abstract to base inheritance
f-PLT May 28, 2025
ee52825
Update .pre-commit-config.yaml
f-PLT Jun 1, 2025
cf437bd
Update pyproject.toml
f-PLT Jun 3, 2025
fe35c4f
Save progress - Prototype url search
f-PLT Jun 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,11 @@ jobs:
if: steps.cache.outputs.cache-hit != 'true'

- name: Install dependencies
env:
CONDA_TOOL: mamba
run: |
make CONDA_TOOL=mamba install
make poetry-install-venv
make install

- name: Run linting checks
run: |
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/precommit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,11 @@ jobs:
if: steps.cache.outputs.cache-hit != 'true'

- name: Install dependencies
env:
CONDA_TOOL: mamba
run: |
make CONDA_TOOL=mamba install
make poetry-install-venv
make install

- name: Run Pre-commit checks
run: |
Expand Down
Loading
Loading