fix: persist inline_data replacement across turns when optimize_data_file=True

[ishanrajsingh]

Summary

Fixes #4013 - Ensures optimize_data_file=True correctly persists inline_data replacement across conversation turns, preventing redundant CSV file transmission to the LLM.

Problem

When optimize_data_file=True was enabled on a custom BaseCodeExecutor, inline_data (CSV files) were replaced with text placeholders (Available file: 'xxx.csv') on the first turn. However, on subsequent turns, the original inline_data reappeared, causing the full CSV to be sent to the LLM again.

Debug Evidence from Issue #4013

=== AFTER _extract_and_replace_inline_files ===
replaced with: Available file: data_1_2.csv
​

=== LLM Request Contents (in before_model_callback) ===
inline_data: mime=text/csv, size=61194 ← inline_data restored!
​

text

Object ID comparison showed different objects between processing steps, confirming the deep copy issue.

Root Cause

1. _extract_and_replace_inline_files() modified llm_request.contents (a temporary copy)
2. session.events (the persistent source) were never updated
3. On each turn, _run_one_step_async() in base_llm_flow.py creates a new LlmRequest via copy.deepcopy(session.events)
4. Since session.events still contained original inline_data, it was restored

Solution

Modified _extract_and_replace_inline_files() to update both:

• llm_request.contents (for current turn)
• session.events (for persistence across turns)

Key Changes in _code_execution.py:

1. Added invocation_context parameter to access session.events
2. Track events needing updates via events_to_update dictionary
3. Match and update session events by comparing inline_data content
4. Use Pydantic's model_copy() instead of dataclasses.replace()
5. Early-exit for placeholders to avoid re-processing

Code Changes:

Before (line ~173)
all_input_files = _extract_and_replace_inline_files(
code_executor_context, llm_request
)

After
all_input_files = _extract_and_replace_inline_files(
code_executor_context, llm_request, invocation_context
)

text

Function now updates session.events directly:
session.events[event_idx] = event.model_copy(
update={'content': updated_content}
)

text

Testing

Added comprehensive test suite in test_code_execution_persistence.py:

Test 1: test_inline_data_replacement_in_extract_function

• Verifies _extract_and_replace_inline_files() modifies both llm_request and session.events
• Confirms inline_data removed and placeholder added
• Validates file extraction

Test 2: test_persistence_across_simulated_turns

• Simulates multi-turn conversation
• Deep copies session.events (like real flow does)
• Verifies inline_data doesn't reappear on turn 2
• Confirms files not re-injected

Test Results:

✓ All new tests pass
✓ Existing code_execution tests pass
✓ No regressions detected

text

Impact

Benefits:

• ✅ Token savings: CSV files sent only once, not on every turn
• ✅ Cost reduction: Lower LLM API costs for multi-turn sessions with files
• ✅ Performance: Reduced request payload size
• ✅ User experience: Faster response times

Risk Assessment:

• ✅ No breaking changes
• ✅ Backwards compatible
• ✅ Only affects optimize_data_file=True code path
• ✅ Existing behavior preserved for optimize_data_file=False

Checklist

• Code follows project style guidelines
• Added comprehensive test coverage
• All tests pass locally
• Updated function signatures and docstrings
• Added debug logging
• Linked issue [BUG]: optimize_data_file=True does not persist inline_data replacement - CSV sent on every LLM call #4013
• No breaking changes
• Tested with both Pydantic Event models

Additional Notes

This fix is particularly important for:

• Sessions with large CSV files (>10KB)
• Long multi-turn conversations
• Users on metered LLM APIs (cost sensitive)
• Enterprise deployments with BaseCodeExecutor subclasses

The issue was reproduced with Claude via Bedrock, confirming it affects production use cases.

[gemini-code-assist]

Summary of Changes

Hello @ishanrajsingh, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical bug in the data optimization flow where inline data, such as CSV files, was not correctly persisted as text placeholders across multiple conversation turns. Previously, enabling optimize_data_file would replace the data with a placeholder for the current turn, but the original data would reappear in subsequent turns, leading to redundant and costly re-transmission to the LLM. The implemented solution ensures that once inline data is optimized, this change is permanently reflected in the session's history, preventing unnecessary data transfer and improving overall efficiency and cost-effectiveness.

Highlights

• Problem Addressed: Fixed an issue where inline data (e.g., CSV files) was redundantly transmitted to the LLM on subsequent conversation turns, even when optimize_data_file=True was enabled. This occurred because the initial replacement of inline data with a text placeholder was not persisted in the session's event history.
• Root Cause Identified: The _extract_and_replace_inline_files() function only modified a temporary copy of the LLM request contents, leaving the original inline_data in session.events unchanged. On new turns, a deep copy of session.events would restore the original inline_data.
• Solution Implemented: The _extract_and_replace_inline_files() function was modified to update both the current llm_request.contents and the persistent session.events. This ensures that once inline data is replaced with a placeholder, that change is maintained across all subsequent turns.
• Key Code Changes: The _extract_and_replace_inline_files function now accepts an invocation_context to access session.events. It tracks and applies updates to session.events using Pydantic's model_copy() and includes an early-exit for already processed placeholders.
• Comprehensive Testing: New tests (test_inline_data_replacement_in_extract_function and test_persistence_across_simulated_turns) were added to verify that inline data is correctly replaced in both the LLM request and session events, and that this replacement persists across simulated multi-turn conversations.
• Impact and Benefits: This fix leads to significant token savings, reduced LLM API costs, improved performance due to smaller request payloads, and a better user experience, especially for sessions involving large files or long multi-turn conversations. It is backwards compatible and only affects the optimize_data_file=True code path.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request effectively addresses a critical bug where inline data was not being persisted correctly across conversation turns when optimize_data_file=True. The solution of updating session.events in addition to llm_request.contents is correct and well-implemented. The addition of a comprehensive test suite is excellent and ensures the fix is robust and prevents future regressions.

My review includes a few suggestions to improve code style, maintainability, and address a potential performance concern. These are mostly minor points and don't detract from the quality of the fix.

[ishanrajsingh]

/gemini review

[gemini-code-assist]

Code Review

This pull request modifies _code_execution.py to ensure that inline data files within LLM requests are replaced with file name placeholders persistently across conversation turns. The _extract_and_replace_inline_files function was updated to accept invocation_context and now modifies both the current llm_request and the invocation_context.session.events to store these placeholders, preventing inline data from reappearing in subsequent interactions. A new test file, test_code_execution_persistence.py, was added to validate this persistence, confirming that inline data is correctly replaced in both the request and session, and that this replacement holds true over multiple simulated turns. The reviewer praised the new persistence test but recommended removing several unused variables to improve its clarity.