Skip to content

fix: HuggingFace backend mot.usage always None without telemetry enabled #694

@planetf1

Description

@planetf1

Bug

ModelOutputThunk.usage is always None for the HuggingFace backend unless telemetry tracing or MELLEA_METRICS_ENABLED=true is set.

Root cause

Introduced in #653 (token metrics hooks refactor). In huggingface.py::_post_process_async, token count extraction was gated behind (span is not None or metrics_enabled), so mot.usage is never populated in plain runs.

Fix

Remove the telemetry guard from token count extraction — usage is a standard mot field, not a telemetry concern. Telemetry reporting further down still checks span/metrics_enabled as before.

# Before
if (span is not None or metrics_enabled) and isinstance(hf_output, GenerateDecoderOnlyOutput):
    ...

# After
if isinstance(hf_output, GenerateDecoderOnlyOutput):
    ...

Impact

test/backends/test_huggingface.py::test_async_avalue fails asserting mot1.usage is not None.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions