Skip to content
This repository was archived by the owner on Jun 5, 2025. It is now read-only.
This repository was archived by the owner on Jun 5, 2025. It is now read-only.

[Story]: Display Prompt cache statistics #940

@kantord

Description

@kantord

Description

Depends on #939 and #892, as until those are fixed the utility of this feature is minimal or none.

Anthropic and OpenAI provide prompt caching features, and their APIs return statistical data about the savings using prompt caching.

As a cost-conscious or environmentally conscious user, I would like to know that I am properly using prompt caching during my daily work, and how much efficiency is gained.

I would also like to be able to see if one of the AI assistants or tools that I am using prevents this feature.

I would like to be able to access this conveniently on the CodeGate dashboard.

Additional Context

We may have to normalize this into a common struct in order to make it usable across providers

For a good starting point on prompt caching in general, you can check out this internal document: https://docs.google.com/document/d/1gEgkIyT1CRcTMAwYiU6ldeeosKKbWMX5HM8AdQx3xV0/edit?tab=t.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions