Skip to content

Migrate model/metadata loading from wandb to HuggingFace#54

Open
avantikalal wants to merge 4 commits into
mainfrom
huggingface
Open

Migrate model/metadata loading from wandb to HuggingFace#54
avantikalal wants to merge 4 commits into
mainfrom
huggingface

Conversation

@avantikalal
Copy link
Copy Markdown
Collaborator

Summary

  • Replaces broken grelu.resources.get_artifact() / wandb-based loading with hf_hub_download from HuggingFace Hub
  • Model weights: Genentech/decima-model, metadata: Genentech/decima-data
  • Preserves full wandb support for internal/private models in new decima.hub.wandb submodule (uses wandb.Api() directly, no gReLU dependency)
  • Drops upper bound on gReLU pin (>=1.0.10)

Changes

File Change
src/decima/constants.py Add HF_MODEL_REPO, HF_DATA_REPO, HF_METADATA_FILENAME
src/decima/hub/__init__.py Rewrite: HuggingFace primary backend, local paths preserved
src/decima/hub/wandb.py New: all wandb functions via wandb.Api() directly
src/decima/hub/download.py Update: hf_hub_download with local_dir
setup.cfg Add huggingface_hub dep, add wandb/hf pytest markers
tests/test_hub.py Update to HF; mark model/metadata tests @pytest.mark.hf
tests/conftest.py Remove login_wandb() call (HF is public)

Internal user migration

# Before
from decima.hub import load_decima_model

# After (for private/internal wandb models)
from decima.hub.wandb import load_decima_model
model = load_decima_model("private_model", host="https://internal.wandb.company.com")

Test plan

  • test_hf_constants — constants correct
  • 39 tests pass with pytest -m "not wandb and not hf and not long_running"
  • test_load_result passes (downloads metadata.h5ad from HF, ~3.3 GB)
  • @pytest.mark.hf tests (full model download) — require HF network access
  • @pytest.mark.wandb tests — require wandb credentials

🤖 Generated with Claude Code

avantikalal and others added 3 commits May 7, 2026 11:38
…t markers

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Add HF_MODEL_REPO, HF_DATA_REPO, HF_METADATA_FILENAME constants
- Rewrite hub/__init__.py to use hf_hub_download (fixes broken grelu.resources import)
- Add hub/wandb.py for internal/private model access via wandb.Api()
- Update hub/download.py to use hf_hub_download with local_dir
- Remove login_wandb() from conftest.py (HF is public, no auth needed)
- Update test_hub.py with hf/wandb markers

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
tox>=4.24 uses dataclass(slots=True) which requires Python 3.10+.
pipx run was downloading latest tox regardless of the pip-installed version.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@avantikalal avantikalal requested a review from MuhammedHasan May 7, 2026 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant