Skip to content

feat(agent): add image generation tool#1594

Merged
zerob13 merged 3 commits intodevfrom
feat/agent-image-generation-tool
May 9, 2026
Merged

feat(agent): add image generation tool#1594
zerob13 merged 3 commits intodevfrom
feat/agent-image-generation-tool

Conversation

@yyhhyyyyyy
Copy link
Copy Markdown
Collaborator

@yyhhyyyyyy yyhhyyyyyy commented May 9, 2026

add image generation tool

iShot_2026-05-09_09.08.32.mp4

Summary by CodeRabbit

  • New Features

    • Configurable Image Generation Model added to agent settings.
    • Built-in agent Image Generation tool for standalone image creation.
    • "Image Generation" tools group and UI labels localized across languages.
  • Bug Fixes

    • Improved image preview handling: richer previews promoted into assistant messages and better preview rendering.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 9, 2026

Review Change Stack
No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b36827f5-e80c-49be-b6d1-8eb5131322bd

📥 Commits

Reviewing files that changed from the base of the PR and between df0efba and 4f188fd.

📒 Files selected for processing (14)
  • src/renderer/src/i18n/da-DK/chat.json
  • src/renderer/src/i18n/fa-IR/chat.json
  • src/renderer/src/i18n/fr-FR/chat.json
  • src/renderer/src/i18n/fr-FR/settings.json
  • src/renderer/src/i18n/he-IL/chat.json
  • src/renderer/src/i18n/he-IL/settings.json
  • src/renderer/src/i18n/ja-JP/chat.json
  • src/renderer/src/i18n/ko-KR/chat.json
  • src/renderer/src/i18n/pt-BR/chat.json
  • src/renderer/src/i18n/pt-BR/settings.json
  • src/renderer/src/i18n/ru-RU/chat.json
  • src/renderer/src/i18n/zh-CN/settings.json
  • src/renderer/src/i18n/zh-HK/settings.json
  • src/renderer/src/i18n/zh-TW/settings.json
✅ Files skipped from review due to trivial changes (12)
  • src/renderer/src/i18n/da-DK/chat.json
  • src/renderer/src/i18n/fa-IR/chat.json
  • src/renderer/src/i18n/ko-KR/chat.json
  • src/renderer/src/i18n/he-IL/settings.json
  • src/renderer/src/i18n/pt-BR/chat.json
  • src/renderer/src/i18n/pt-BR/settings.json
  • src/renderer/src/i18n/ja-JP/chat.json
  • src/renderer/src/i18n/zh-TW/settings.json
  • src/renderer/src/i18n/ru-RU/chat.json
  • src/renderer/src/i18n/fr-FR/settings.json
  • src/renderer/src/i18n/he-IL/chat.json
  • src/renderer/src/i18n/zh-HK/settings.json
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/renderer/src/i18n/zh-CN/settings.json
  • src/renderer/src/i18n/fr-FR/chat.json

📝 Walkthrough

Walkthrough

This PR adds standalone image generation for DeepChat agents: presenter streaming, a new agent MCP tool, image-preview promotion into assistant messages, agent config and settings UI support (with i18n), runtime wiring, and tests.

Changes

Standalone Image Generation Feature

Layer / File(s) Summary
Type Definitions & Contracts
src/shared/agentImageGenerationTool.ts, src/shared/contracts/domainSchemas.ts, src/shared/types/agent-interface.d.ts, src/shared/types/presenters/llmprovider.presenter.d.ts, src/shared/types/presenters/legacy.presenters.d.ts
New tool/server name constants, imageGenerationModel in DeepChat config, StandaloneImageGenerationResult type, and generateImageStandalone presenter signature.
LLM Provider Implementation
src/main/presenter/llmProviderPresenter/index.ts
Adds generateImageStandalone(providerId, prompt, modelId, imageOptions?, options?) with abort support, rate limiting, streaming image collection, and result shaping.
Agent Image Generation Tool
src/main/presenter/toolPresenter/agentTools/agentImageGenerationTool.ts
MCP tool with Zod validation, canUse, model resolution, call() invoking presenter image generation, preview metadata conversion, and recoverable error envelopes.
Image Preview Presentation
src/main/presenter/agentRuntimePresenter/imageGenerationBlocks.ts
prepareToolImagePreviewPresentation conditionally promotes previews to assistant image blocks; insertBlocksAfterToolCall inserts promoted blocks after tool calls.
Tool Dispatch & Finalization
src/main/presenter/agentRuntimePresenter/dispatch.ts
Staged tool results now include toolSource/serverName; finalization uses presentation utilities to populate toolBlock previews and insert promoted assistant image blocks.
Agent Runtime Coordination
src/main/presenter/agentRuntimePresenter/index.ts, src/main/presenter/index.ts
Deferred tool execution results include tool metadata; respondToolInteraction applies image presentation and inserts promoted blocks; presenter adapter correctly exposes generateImageStandalone.
Tool Manager & Presenter Integration
src/main/presenter/toolPresenter/agentTools/agentToolManager.ts, src/main/presenter/toolPresenter/agentTools/index.ts, src/main/presenter/toolPresenter/index.ts, src/main/presenter/toolPresenter/runtimePorts.ts
Instantiates image tool, reserves image_generate name in ToolPresenter, adds image tool guidance in the system prompt, and includes generateImageStandalone in runtime port.
Configuration Support
src/main/presenter/agentRepository/index.ts, src/main/presenter/configPresenter/index.ts
mergeDeepChatConfig now merges imageGenerationModel; deprecated image model selections are cleared in cleanup.
Settings UI
src/renderer/settings/components/DeepChatAgentsSettings.vue
Adds imageGenerationModel field, popover state, typed ModelSelect usage, form binding, and persistence under config.imageGenerationModel; updates tool group ordering.
Localization
`src/renderer/src/i18n/*/(settings
chat).json`
Tests
test/main/presenter/*, test/renderer/components/*
New and updated tests for config merging, generateImageStandalone streaming, agent image tool execution, dispatch preview promotion, tool manager/presenter mocks, and settings UI.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested reviewers

  • zerob13

Poem

🐰 I hopped through code and sketched light,

From prompt to pixel, morning bright,
Blocks promoted, previews bloom,
Agents paint within the room,
Carrot-powered art takes flight!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'feat(agent): add image generation tool' directly and clearly describes the main change: adding a new image generation tool to the agent system.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/agent-image-generation-tool

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/renderer/settings/components/DeepChatAgentsSettings.vue (1)

716-723: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add a localized label mapping for agent-image-generation tool group.

GROUP_ORDER includes agent-image-generation, but getGroupLabel has no matching case, so the UI can fall back to raw group key text.

💡 Proposed fix
 const getGroupLabel = (serverName: string) => {
   switch (serverName) {
     case 'agent-filesystem':
       return t('chat.input.tools.groups.agentFilesystem')
     case 'agent-core':
       return t('chat.input.tools.groups.agentCore')
+    case 'agent-image-generation':
+      return t('chat.input.tools.groups.agentImageGeneration')
     case 'agent-skills':
       return t('chat.input.tools.groups.agentSkills')
     case 'deepchat-settings':
       return t('chat.input.tools.groups.deepchatSettings')
     case 'yobrowser':
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/renderer/settings/components/DeepChatAgentsSettings.vue` around lines 716
- 723, getGroupLabel is missing a mapping for the 'agent-image-generation' key
from GROUP_ORDER, causing the raw group key to display; update the getGroupLabel
function to return a localized label for 'agent-image-generation' (using the
same translation function/locale pattern as other cases) so the UI shows a
human-friendly name for that tool group.
🧹 Nitpick comments (1)
test/main/presenter/toolPresenter/toolPresenter.test.ts (1)

49-82: ⚡ Quick win

Consider extracting a shared agentToolRuntime factory to reduce duplication.

The agentToolRuntime mock object (including the newly added generateImageStandalone: vi.fn()) is verbatim copy-pasted across all six test cases. A single factory function or a beforeEach setup would make future additions (like a new method on ILlmProviderPresenter) a one-line change instead of six.

♻️ Suggested refactor: extract a factory
+const buildAgentToolRuntime = () => ({
+  resolveConversationWorkdir: vi.fn().mockResolvedValue(null),
+  resolveConversationSessionInfo: vi.fn().mockResolvedValue(null),
+  getSkillPresenter: () =>
+    ({
+      getActiveSkills: vi.fn().mockResolvedValue([]),
+      getActiveSkillsAllowedTools: vi.fn().mockResolvedValue([]),
+      listSkillScripts: vi.fn().mockResolvedValue([]),
+      getSkillExtension: vi.fn().mockResolvedValue({
+        version: 1,
+        env: {},
+        runtimePolicy: { python: 'auto', node: 'auto' },
+        scriptOverrides: {}
+      })
+    }) as any,
+  getYoBrowserToolHandler: () => ({
+    getToolDefinitions: vi.fn().mockReturnValue([]),
+    callTool: vi.fn()
+  }),
+  getFilePresenter: () => ({
+    getMimeType: vi.fn(),
+    prepareFileCompletely: vi.fn()
+  }),
+  getLlmProviderPresenter: () => ({
+    executeWithRateLimit: vi.fn().mockResolvedValue(undefined),
+    generateCompletionStandalone: vi.fn(),
+    generateImageStandalone: vi.fn()
+  }),
+  createSettingsWindow: vi.fn(),
+  sendToWindow: vi.fn().mockReturnValue(true),
+  getApprovedFilePaths: vi.fn().mockReturnValue([]),
+  consumeSettingsApproval: vi.fn().mockReturnValue(false)
+})

 // Then each test uses:
-      agentToolRuntime: {
-        resolveConversationWorkdir: vi.fn().mockResolvedValue(null),
-        // ... 15 more lines ...
-      }
+      agentToolRuntime: buildAgentToolRuntime()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/main/presenter/toolPresenter/toolPresenter.test.ts` around lines 49 -
82, Extract a shared factory for the repeated mock object to remove duplication:
create a helper (e.g., buildAgentToolRuntimeMock) that returns the
agentToolRuntime object used in toolPresenter.test.ts and include
getLlmProviderPresenter with generateImageStandalone, getYoBrowserToolHandler,
getSkillPresenter, getFilePresenter, createSettingsWindow, sendToWindow,
getApprovedFilePaths, consumeSettingsApproval, etc.; then replace the six inline
copies with calls to buildAgentToolRuntimeMock (or set it in a beforeEach) so
adding a new ILlmProviderPresenter method requires only a single edit in the
factory.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/main/presenter/llmProviderPresenter/index.ts`:
- Around line 55-80: The createAbortPromise function must synchronously handle a
signal that is already aborted before attaching the listener: inside
createAbortPromise (and before adding the 'abort' event listener) check
signal.aborted and if true call onAbort() and return an already-rejected promise
(rejecting with createAbortError()) along with a no-op cleanup; otherwise
proceed to attach the listener as currently implemented and keep the existing
cleanup that removes the listener. Ensure you reference createAbortPromise and
the onAbort callback so callers (e.g., the stream return invocation) get invoked
immediately when signal.aborted is already true.

In `@src/renderer/src/i18n/zh-CN/settings.json`:
- Line 117: The translation key deepchatAgents.imageGenerationModel currently
uses the informal term "生图模型" which is inconsistent with
model.modelConfig.type.options.imageGeneration ("图像生成模型"); update the value of
deepchatAgents.imageGenerationModel to "图像生成模型" so both keys
(deepchatAgents.imageGenerationModel and
model.modelConfig.type.options.imageGeneration) use the same formal phrase
across zh-CN.

---

Outside diff comments:
In `@src/renderer/settings/components/DeepChatAgentsSettings.vue`:
- Around line 716-723: getGroupLabel is missing a mapping for the
'agent-image-generation' key from GROUP_ORDER, causing the raw group key to
display; update the getGroupLabel function to return a localized label for
'agent-image-generation' (using the same translation function/locale pattern as
other cases) so the UI shows a human-friendly name for that tool group.

---

Nitpick comments:
In `@test/main/presenter/toolPresenter/toolPresenter.test.ts`:
- Around line 49-82: Extract a shared factory for the repeated mock object to
remove duplication: create a helper (e.g., buildAgentToolRuntimeMock) that
returns the agentToolRuntime object used in toolPresenter.test.ts and include
getLlmProviderPresenter with generateImageStandalone, getYoBrowserToolHandler,
getSkillPresenter, getFilePresenter, createSettingsWindow, sendToWindow,
getApprovedFilePaths, consumeSettingsApproval, etc.; then replace the six inline
copies with calls to buildAgentToolRuntimeMock (or set it in a beforeEach) so
adding a new ILlmProviderPresenter method requires only a single edit in the
factory.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c71a5593-d9d9-4ddd-86ab-ddc48852da5b

📥 Commits

Reviewing files that changed from the base of the PR and between 61abd81 and 2ce9866.

📒 Files selected for processing (44)
  • src/main/presenter/agentRepository/index.ts
  • src/main/presenter/agentRuntimePresenter/dispatch.ts
  • src/main/presenter/agentRuntimePresenter/imageGenerationBlocks.ts
  • src/main/presenter/agentRuntimePresenter/index.ts
  • src/main/presenter/configPresenter/index.ts
  • src/main/presenter/index.ts
  • src/main/presenter/llmProviderPresenter/index.ts
  • src/main/presenter/toolPresenter/agentTools/agentImageGenerationTool.ts
  • src/main/presenter/toolPresenter/agentTools/agentToolManager.ts
  • src/main/presenter/toolPresenter/agentTools/index.ts
  • src/main/presenter/toolPresenter/index.ts
  • src/main/presenter/toolPresenter/runtimePorts.ts
  • src/renderer/settings/components/DeepChatAgentsSettings.vue
  • src/renderer/src/i18n/da-DK/settings.json
  • src/renderer/src/i18n/en-US/settings.json
  • src/renderer/src/i18n/fa-IR/settings.json
  • src/renderer/src/i18n/fr-FR/settings.json
  • src/renderer/src/i18n/he-IL/settings.json
  • src/renderer/src/i18n/ja-JP/settings.json
  • src/renderer/src/i18n/ko-KR/settings.json
  • src/renderer/src/i18n/pt-BR/settings.json
  • src/renderer/src/i18n/ru-RU/settings.json
  • src/renderer/src/i18n/zh-CN/settings.json
  • src/renderer/src/i18n/zh-HK/settings.json
  • src/renderer/src/i18n/zh-TW/settings.json
  • src/shared/agentImageGenerationTool.ts
  • src/shared/contracts/domainSchemas.ts
  • src/shared/types/agent-interface.d.ts
  • src/shared/types/presenters/index.d.ts
  • src/shared/types/presenters/legacy.presenters.d.ts
  • src/shared/types/presenters/llmprovider.presenter.d.ts
  • test/main/presenter/agentRepository.test.ts
  • test/main/presenter/agentRuntimePresenter/agentRuntimePresenter.test.ts
  • test/main/presenter/agentRuntimePresenter/dispatch.test.ts
  • test/main/presenter/llmProviderPresenter.test.ts
  • test/main/presenter/toolPresenter/agentTools/agentImageGenerationTool.test.ts
  • test/main/presenter/toolPresenter/agentTools/agentToolManagerRead.test.ts
  • test/main/presenter/toolPresenter/agentTools/agentToolManagerSettings.test.ts
  • test/main/presenter/toolPresenter/agentTools/agentToolManagerSkillAccess.test.ts
  • test/main/presenter/toolPresenter/agentTools/subagentOrchestratorTool.test.ts
  • test/main/presenter/toolPresenter/toolPresenter.test.ts
  • test/main/routes/contracts.test.ts
  • test/renderer/components/DeepChatAgentsSettings.test.ts
  • test/renderer/components/message/MessageBlockMedia.test.ts

Comment on lines +55 to +80
const createAbortPromise = (
signal: AbortSignal | undefined,
onAbort?: () => void
): { promise?: Promise<never>; cleanup: () => void } => {
if (!signal) {
return { cleanup: () => undefined }
}

let abortHandler: (() => void) | null = null
const promise = new Promise<never>((_, reject) => {
abortHandler = () => {
onAbort?.()
reject(createAbortError())
}
signal.addEventListener('abort', abortHandler, { once: true })
})

return {
promise,
cleanup: () => {
if (abortHandler) {
signal.removeEventListener('abort', abortHandler)
}
}
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

sed -n '55,80p' src/main/presenter/llmProviderPresenter/index.ts
printf '\n---\n'
sed -n '382,412p' src/main/presenter/llmProviderPresenter/index.ts

Repository: ThinkInAIXYZ/deepchat

Length of output: 1711


Check signal.aborted before attaching the abort listener in createAbortPromise.

The function checks for a signal and attaches an addEventListener listener, but if the signal becomes aborted between the early check at line 383 and the listener attachment at line 401, the listener will never fire because AbortSignal events only notify newly-added listeners. The code will hang on the stream operation despite the caller having cancelled.

Add a synchronous signal.aborted check inside createAbortPromise immediately after (or before) attaching the listener to handle signals that abort during the setup window:

Current code structure
// Line 383-385: Early check (insufficient)
if (signal?.aborted) {
  throw createAbortError()
}

// Lines 390-397: Start stream
const stream = provider.coreStream(...)

// Lines 400-402: Attach listener (too late if signal aborts here)
const abort = createAbortPromise(signal, () => {
  void stream.return?.(undefined as never)
})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/main/presenter/llmProviderPresenter/index.ts` around lines 55 - 80, The
createAbortPromise function must synchronously handle a signal that is already
aborted before attaching the listener: inside createAbortPromise (and before
adding the 'abort' event listener) check signal.aborted and if true call
onAbort() and return an already-rejected promise (rejecting with
createAbortError()) along with a no-op cleanup; otherwise proceed to attach the
listener as currently implemented and keep the existing cleanup that removes the
listener. Ensure you reference createAbortPromise and the onAbort callback so
callers (e.g., the stream return invocation) get invoked immediately when
signal.aborted is already true.

Comment thread src/renderer/src/i18n/zh-CN/settings.json Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/renderer/src/i18n/fr-FR/chat.json`:
- Line 52: The French locale value for the key "agentImageGeneration" is
currently English; replace the English string with a proper French translation
(for example "Génération d’images") in the fr-FR chat JSON so the UI is fully
localized; locate the "agentImageGeneration" entry in the fr-FR/chat.json and
update its value accordingly, preserving the surrounding JSON structure and
quotation style.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3ddbb6bb-bc54-423a-975a-b17d7df11fca

📥 Commits

Reviewing files that changed from the base of the PR and between 2ce9866 and df0efba.

📒 Files selected for processing (14)
  • src/renderer/settings/components/DeepChatAgentsSettings.vue
  • src/renderer/src/i18n/da-DK/chat.json
  • src/renderer/src/i18n/en-US/chat.json
  • src/renderer/src/i18n/fa-IR/chat.json
  • src/renderer/src/i18n/fr-FR/chat.json
  • src/renderer/src/i18n/he-IL/chat.json
  • src/renderer/src/i18n/ja-JP/chat.json
  • src/renderer/src/i18n/ko-KR/chat.json
  • src/renderer/src/i18n/pt-BR/chat.json
  • src/renderer/src/i18n/ru-RU/chat.json
  • src/renderer/src/i18n/zh-CN/chat.json
  • src/renderer/src/i18n/zh-HK/chat.json
  • src/renderer/src/i18n/zh-TW/chat.json
  • test/main/presenter/toolPresenter/toolPresenter.test.ts
✅ Files skipped from review due to trivial changes (11)
  • src/renderer/src/i18n/en-US/chat.json
  • src/renderer/src/i18n/zh-TW/chat.json
  • src/renderer/src/i18n/da-DK/chat.json
  • src/renderer/src/i18n/pt-BR/chat.json
  • src/renderer/src/i18n/zh-HK/chat.json
  • src/renderer/src/i18n/ko-KR/chat.json
  • src/renderer/src/i18n/ja-JP/chat.json
  • src/renderer/src/i18n/zh-CN/chat.json
  • src/renderer/src/i18n/fa-IR/chat.json
  • src/renderer/src/i18n/ru-RU/chat.json
  • src/renderer/src/i18n/he-IL/chat.json
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/main/presenter/toolPresenter/toolPresenter.test.ts
  • src/renderer/settings/components/DeepChatAgentsSettings.vue

Comment thread src/renderer/src/i18n/fr-FR/chat.json Outdated
@zerob13 zerob13 merged commit 42599e6 into dev May 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants