feat: local telemetry storage

[fatih-acar]

• I have reviewed AI generated content

As per the quickstart.md file:

What This Feature Does

Infrahub now stores a daily telemetry snapshot locally in the database, regardless of whether remote telemetry reporting is enabled. This ensures all deployments — including air-gapped and opted-out — retain usage data for support, auditing, and license compliance.

Key Changes

1. Automatic daily storage: The daily telemetry workflow now always stores a snapshot locally before optionally sending to the remote endpoint.
2. CLI export: infrahubctl telemetry export exports snapshots to a JSON file.
3. CLI listing: infrahubctl telemetry list shows a summary table of stored snapshots.
4. REST API: GET /api/telemetry/snapshots provides programmatic access.
5. Permission: Access requires READ_TELEMETRY global permission.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (4)

• main
• stable
• develop
• release-.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[dgarros]

I think we need to be careful with that, overtime the format of this data will change and we'll have old data in the database, we need to make sure that we can still read and export the old data, not just the latest one

[fatih-acar]

Good point... We should probably create a TelemetryData Pydantic class for each format version, so that it gets correctly (de)serialized. I will add to the plan.

[fatih-acar]

Updated the spec: instead of versioned Pydantic classes, the data field is now stored as plain dict[str, Any] (JSON). The TelemetryData Pydantic model is still used at collection time for validation, then model.model_dump() converts it to a plain dict before storage. On read, data comes back as a raw dict — no deserialization needed. The payload_format field identifies which version of the schema produced the data, so consumers can interpret differences if needed. This keeps old snapshots always readable regardless of how the model evolves.

[dgarros]

it's not clearly mentioned but I think we should have a sane default limit for this API to avoid returning everything each time and potentially having a pagination system would be helpful to avoid returning 100s of responses at once

[fatih-acar]

Pagination system will be implemented as described in contracts/rest-api.md‎

[github-actions]

This PR has been inactive for 2 weeks and has been marked as stale.

If you're still working on this, please:

• Push new commits, or
• Leave a comment to indicate this PR is still active

The stale label will be removed automatically when there's new activity.

[ajtmccarty]

looks pretty good to me

[ajtmccarty]

might be nice for this to be structured data too, but not sure if it really matters in this case
I see the research section explains why the data is unstructured: to be backwards compatible with old payload formats when updating the format going forward

[fatih-acar]

Thanks for the review @ajtmccarty I've addressed your comments and pushed the actual implementation
I'll do some manual testing

[codspeed-hq]

Merging this PR will not alter performance

✅ 12 untouched benchmarks

---

_{Comparing fac-001-local-telemetry-storage (6151156) with release-1.9 (d34952a)}

[cubic-dev-ai]

4 issues found across 23 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="dev/specs/fac-001-local-telemetry-storage/contracts/rest-api.md">

<violation number="1" location="dev/specs/fac-001-local-telemetry-storage/contracts/rest-api.md:75">
P2: Type `remote_send_status` as `Literal["sent", "skipped", "failed", "pending"]` (or a `StrEnum`) instead of plain `str`. This gives Pydantic validation and self-documents the allowed values in the contract.

(Based on your team's feedback about preferring enums and typed values over literals.) [FEEDBACK_USED]</violation>

<violation number="2" location="dev/specs/fac-001-local-telemetry-storage/contracts/rest-api.md:121">
P1: The `export` command lacks `--limit`/`--offset` options and there's no mention of automatic pagination. Since the REST API defaults to `limit=1000`, an export from a long-running deployment would silently truncate results. Either add explicit pagination options (matching `telemetry list`) or specify that the command auto-paginates through all results.</violation>
</file>

<file name="backend/infrahub/telemetry/tasks.py">

<violation number="1" location="backend/infrahub/telemetry/tasks.py:126">
P2: `database` is only assigned inside the first `try` block. If `get_database()` fails, subsequent blocks reference an unbound `database` variable. Move `database = await get_database()` before the first `try`, or re-obtain it in each block that needs it, so the variable is always defined when used.</violation>
</file>

<file name="backend/infrahub/api/telemetry.py">

<violation number="1" location="backend/infrahub/api/telemetry.py:83">
P2: `count` returns the current page size, not the total number of matching snapshots. For a paginated endpoint (with `limit`/`offset`), this makes it impossible for clients to determine whether more pages exist. Either rename this to clarify it's the page size (e.g., `returned`), or add a total-count query and return the true total here.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}

[polmichel]

This looks good! Just a comment worth really looking into regarding the end_date bound in the query when querying "today's data

[ajtmccarty]

I was just on my dependency injection soapbox yesterday for a different PR making some changes to a flow and I'm going to do it again

to be more SOLID (particularly the D for dependency injection), this flow should just instantiate a new TelemetrySnapshotService that encapsulates all of the below logic. something like this

async def send_telemetry_push() -> None:
    telemetry_repository = TelemetryRepository(db=db)
    telemetry_service = TelemetrySnapshotService(
        db=db, telemetry_repository=telemetry_repository,
    )

    data = await gather_anonymous_telemetry_data()
    await telemetry_service.save_and_send(data)

and then in your tests, you don't have to patch a bunch of stuff. you can create mocks using telemetry_repo = AsyncMock(spec=TelemetryRepository) or create_autospec(TelemetryRepository, spec_set=True) and you can assert they are called with telemetry_repo.assert_awaited_once_with() or telemetry_repo.assert_has_awaits() or another method on the mock like this

I think it improves tests (accurate interfaces for the mocks instead of "anything goes") and makes them easier to read (no more patch context managers)

that said, this would probably require some more refactoring to turn gather_anonymous_telemetry_data into TelemetryDataGatherer and post_telemetry_data into TelemetryDataPoster to get the full benefits, but something to consider and an LLM can probably make pretty quick work of it

[fatih-acar]

Thanks I'll try to tackle that refactor in another PR (to not block 1.9)

[cubic-dev-ai]

1 issue found across 12 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="backend/infrahub/telemetry/tasks.py">

<violation number="1" location="backend/infrahub/telemetry/tasks.py:127">
P1: Unhandled exception: `repository.save()` on the opt-out path lost its `try/except` during refactoring. The other two `repository.save()` calls in this function are protected, but a DB failure here will crash the flow. Wrap this in the same error-handling pattern used by the adjacent save calls.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}