Skip to content

add FileContent and ToolOutput#5087

Open
tinalenguyen wants to merge 8 commits intomainfrom
tina/support-image-content-tool-output
Open

add FileContent and ToolOutput#5087
tinalenguyen wants to merge 8 commits intomainfrom
tina/support-image-content-tool-output

Conversation

@tinalenguyen
Copy link
Member

@tinalenguyen tinalenguyen commented Mar 11, 2026

  • backward compatible, does not break existing user code assuming that tool outputs are str
  • add more depth to function tool outputs, allowing for ImageContent and FileContent to be passed along with strings, like so:
    @function_tool
    async def get_chart(self, context: RunContext):
        """Returns a chart image and checkup report when the user asks for a chart or visual."""
        with open("mock_checkup_report.pdf", "rb") as f:
            report_data = f.read()
        img_content = ImageContent(image="https://my.clevelandclinic.org/-/scassets/images/org/health/articles/10881-vital-signs")
        file_content = FileContent(name="mock_checkup_report.pdf", data=report_data, mime_type="application/pdf"),
        return "Here is the normal adult vital chart:", img_content, "And the patient report file:", file_content
            

@tinalenguyen tinalenguyen linked an issue Mar 11, 2026 that may be closed by this pull request
name: str = Field(default="")
call_id: str
output: str
output: ToolOutput
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't this just be?
ToolOutput = ImageContent | FileContent | str

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if users wanted to pass multiple items from a function tool

@tinalenguyen tinalenguyen marked this pull request as ready for review March 18, 2026 18:30
@chenghao-mou chenghao-mou requested a review from a team March 18, 2026 18:30
devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: support ImageContent in tool return value

2 participants