Updates to pipeline & tools for 2026#74

Draft

mihow wants to merge 15 commits intocopilot/migrate-poetry-to-uvfrom

melabbas/pipeline-and-tools

Commits on Mar 8, 2026

chore: exclude data, plans, and SLURM log files from version control

mohamedelabbas1996
and
claude
committed
config: add GBIF credentials env vars and fiftyone optional dependency

mohamedelabbas1996
and
claude
committed
chore: update lockfiles after adding fiftyone optional dependency

mohamedelabbas1996
and
claude
committed
chore: standardize SLURM headers and venv activation in existing job scripts

mohamedelabbas1996
and
claude
committed
feat: include verbatimScientificName in DwCA data loading

mohamedelabbas1996
and
claude
committed
feat: allow clean-dataset to run without prior verification step

mohamedelabbas1996
and
claude
committed
feat: add download-gbif module and CLI command for GBIF DwCA downloads

mohamedelabbas1996
and
claude
committed
feat: add fetch-and-pack module for quota-safe chunked webdataset creation

mohamedelabbas1996
and
claude
committed
feat: add FiftyOne webdataset visualization module

mohamedelabbas1996
and
claude
committed
feat: add Vermont species pipeline SLURM scripts

mohamedelabbas1996
and
claude
committed
feat: add GBIF download and DwCA inspection job scripts

mohamedelabbas1996
and
claude
committed
feat: add FiftyOne visualization job scripts

mohamedelabbas1996
and
claude
committed
feat: add fetch-and-pack chunked pipeline and test scripts

mohamedelabbas1996
and
claude
committed
chore: add archived and legacy pipeline scripts

mohamedelabbas1996
and
claude
committed
feat: add webdataset inspection script with class distribution stats
mohamedelabbas1996
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updates to pipeline & tools for 2026#74

Updates to pipeline & tools for 2026#74
mihow wants to merge 15 commits intocopilot/migrate-poetry-to-uvfrom
melabbas/pipeline-and-tools

Commits on Mar 8, 2026

chore: exclude data, plans, and SLURM log files from version control

config: add GBIF credentials env vars and fiftyone optional dependency

chore: update lockfiles after adding fiftyone optional dependency

chore: standardize SLURM headers and venv activation in existing job scripts

feat: include verbatimScientificName in DwCA data loading

feat: allow clean-dataset to run without prior verification step

feat: add download-gbif module and CLI command for GBIF DwCA downloads

feat: add fetch-and-pack module for quota-safe chunked webdataset creation

feat: add FiftyOne webdataset visualization module

feat: add Vermont species pipeline SLURM scripts

feat: add GBIF download and DwCA inspection job scripts

feat: add FiftyOne visualization job scripts

feat: add fetch-and-pack chunked pipeline and test scripts

chore: add archived and legacy pipeline scripts

feat: add webdataset inspection script with class distribution stats