[Investigation]: investigate Prompt Caching and how it could benefit CodeGate

[davolokh]

Description

Take a deeper look into Prompts Caching feature of Anthropic and check if CodeGate could benefit from them.

Additional Context

Some point from the call:

How many clients use this?
How does cline do this?
Do those using it have shortcomings?
How would this work within CodeGate
How does this compare to RAG?
Can prompt cache work within MUXing?
[Jakub] Cursor has a related files thing, is that the same?
Is there something extra we could do, to make PC’ing even more useful (for example we see the same files in the prompt)
Build up a cache of common system prompts , save the whole thing being 

Useful links:

• https://github.com/continuedev/prompt-file-examples/blob/main/claude-prompt-caching.md?ref=blog.continue.dev
• https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
• https://platform.openai.com/docs/guides/prompt-caching

[kantord]

So far @alex-mcgovern and I identified this issue related to prompt caching: #892

This basically prevents Anthropic prompt caching from working at the moment

[lukehinds]

Thanks @kantord , still worth doing a round up on how others are using this and what extra value would we bring.

[alex-mcgovern]

Summary of our findings here