Skip to content

Conversation

@weinibuliu
Copy link
Member

@weinibuliu weinibuliu commented Dec 1, 2025

#141

Summary by Sourcery

添加对使用自定义内存图像控制器和 UI 工作流上传图像作为匹配目标屏幕的支持。

新功能:

  • 引入一个自定义控制器实现,将上传的图像用作截图来源。
  • 在 MaaFW 中暴露新的连接方法,以初始化并使用基于自定义图像的控制器。
  • 扩展 Web UI,新增 “Custom” 控制器选项卡,允许用户上传图像文件以驱动匹配任务。

增强:

  • 在使用自定义控制器时禁用操作和点击,使任务仅执行图像匹配而不产生副作用。
Original summary in English

Summary by Sourcery

Add support for using a custom in-memory image controller and UI workflow to upload an image as the target screen for matching.

New Features:

  • Introduce a custom controller implementation that uses an uploaded image as the screenshot source.
  • Expose a new connection method in MaaFW to initialize and use the custom image-based controller.
  • Extend the web UI with a new "Custom" controller tab that lets users upload an image file to drive matching tasks.

Enhancements:

  • Disable actions and clicks when using the custom controller so that tasks only perform image matching without side effects.

@sourcery-ai
Copy link

sourcery-ai bot commented Dec 1, 2025

Reviewer's Guide

添加对使用已上传图像作为自定义控制器源的支持,通过 MaaFW 和 Web UI 进行串联,并在使用该自定义控制器时禁用具有副作用的动作和点击。

上传图像并连接自定义控制器的时序图

sequenceDiagram
    actor User
    participant WebUI as WebUI_master_control
    participant MaaFW
    participant MyCustomController
    participant Tasker
    participant AgentClient

    User->>WebUI: Select Custom tab
    User->>WebUI: Upload image file
    WebUI->>WebUI: on_upload(e)
    WebUI->>MaaFW: connect_custom_controller(e.content.read())
    activate MaaFW
    MaaFW->>MyCustomController: new(img_bytes)
    activate MyCustomController
    MyCustomController-->>MaaFW: instance with ndarray
    MaaFW->>MyCustomController: post_connection()
    MyCustomController-->>MaaFW: Future(wait).succeeded
    deactivate MyCustomController
    MaaFW-->>WebUI: True, None
    deactivate MaaFW
    WebUI-->>User: Show controller connected status

    User->>WebUI: Start task
    WebUI->>MaaFW: run_task(entry, pipeline_override)
    activate MaaFW
    MaaFW->>MaaFW: isinstance(controller, CustomController)
    MaaFW->>MaaFW: pipeline_override.update(entry -> DoNothing)
    MaaFW->>AgentClient: register_sink(resource, controller, tasker)
    AgentClient-->>MaaFW: bool
    MaaFW->>Tasker: post_task(entry, pipeline_override)
    Tasker-->>MaaFW: Future(wait).succeeded
    MaaFW-->>WebUI: succeeded, None
    deactivate MaaFW
    WebUI-->>User: Show task result without actions
Loading

带有 MyCustomController 和自定义控制器串联的 MaaFW 类图

classDiagram
    class CustomController {
        <<abstract>>
        +post_connection() Future
        +post_click(x, y) Future
        +screencap() ndarray
        +connect() bool
        +request_uuid() str
    }

    class MyCustomController {
        +MyCustomController(img_bytes)
        +ndarray ndarray
        +connect() bool
        +request_uuid() str
        +screencap() ndarray
    }

    class AdbController {
        +post_connection() Future
        +post_click(x, y) Future
        +screencap() ndarray
        +connect() bool
        +request_uuid() str
    }

    class Win32Controller {
        +post_connection() Future
        +post_click(x, y) Future
        +screencap() ndarray
        +connect() bool
        +request_uuid() str
    }

    class Resource
    class Tasker {
        +post_task(entry, pipeline_override) Future
        +stop() void
    }

    class AgentClient {
        +register_sink(resource, controller, tasker) bool
    }

    class MaaFW {
        +resource Resource
        +controller AdbController
        +controller Win32Controller
        +controller CustomController
        +tasker Tasker
        +agent AgentClient
        +connect_custom_controller(img_bytes) Tuple~bool, str~
        +run_task(entry, pipeline_override) Tuple~bool, str~
        +click(x, y) bool
    }

    CustomController <|-- MyCustomController
    CustomController <|-- AdbController
    CustomController <|-- Win32Controller

    MaaFW --> Resource
    MaaFW --> CustomController
    MaaFW --> Tasker
    MaaFW --> AgentClient
Loading

文件级变更

Change Details Files
引入一个基于 CustomController 的实现,从上传的图像提供帧,并将其作为新的控制器类型接入 MaaFW。
  • 导入 numpy,并定义 MyCustomController,将上传的图像字节转换为 NumPy 数组,并从 screencap 返回该数组。
  • 扩展 MaaFW.controller 的类型注解以包含 CustomController。
  • 新增 MaaFW.connect_custom_controller,用图像字节实例化 MyCustomController,并执行连接后的初始化。
src/MaaDebugger/maafw/__init__.py
在使用自定义控制器时调整任务执行和点击处理行为,以避免执行真实动作。
  • 在 MaaFW.run_task 中检测 CustomController,并在发布任务前将 pipeline 的入口重写为 DoNothing 动作。
  • 在 MaaFW.click 中,当 controller 为 CustomController 时提前返回 False,以阻止分发点击。
src/MaaDebugger/maafw/__init__.py
在 Web UI 中暴露新的 “Custom” 控制器选项,使用户可以通过上传图像建立自定义控制器连接。
  • 在控制器连接界面中添加 “Custom” 选项卡及其对应的选项卡面板串联。
  • 实现 connect_custom_control:上传图像,使用文件内容调用 maafw.connect_custom_controller,更新 GlobalStatus,并在失败时展示错误通知。
  • 将自定义控制器选项卡与现有的 Adb 和 Win32 一起绑定到 controller_type 存储中。
src/MaaDebugger/webpage/index_page/master_control.py

可能关联的问题


提示与命令

与 Sourcery 交互

  • 触发新评审: 在 pull request 中评论 @sourcery-ai review
  • 继续讨论: 直接回复 Sourcery 的评审评论。
  • 从评审评论生成 GitHub issue: 在评审评论下回复,请求 Sourcery 从该评论创建 issue。你也可以在评审评论中回复 @sourcery-ai issue 来从中创建 issue。
  • 生成 pull request 标题: 在 pull request 标题的任意位置写上 @sourcery-ai,即可随时生成标题。你也可以在 pull request 中评论 @sourcery-ai title 来(重新)生成标题。
  • 生成 pull request 摘要: 在 pull request 正文的任意位置写上 @sourcery-ai summary,即可在该位置生成 PR 摘要。你也可以在 pull request 中评论 @sourcery-ai summary 来(重新)生成摘要。
  • 生成审阅者指南: 在 pull request 中评论 @sourcery-ai guide,即可随时(重新)生成审阅者指南。
  • 解决所有 Sourcery 评论: 在 pull request 中评论 @sourcery-ai resolve,可将所有 Sourcery 评论标记为已解决。如果你已经处理完所有评论且不再希望看到它们,这将非常有用。
  • 撤销所有 Sourcery 评审: 在 pull request 中评论 @sourcery-ai dismiss,可撤销所有现有的 Sourcery 评审。尤其适合你希望从一个全新的评审开始的场景 —— 别忘了再评论 @sourcery-ai review 以触发新评审!

自定义你的体验

访问你的 dashboard 来:

  • 启用或禁用评审功能,例如 Sourcery 自动生成的 pull request 摘要、审阅者指南等。
  • 更改评审语言。
  • 添加、删除或编辑自定义评审说明。
  • 调整其他评审设置。

获取帮助

Original review guide in English

Reviewer's Guide

Adds support for using an uploaded image as a custom controller source, wiring it through MaaFW and the web UI, and disables side-effectful actions and clicks when operating with this custom controller.

Sequence diagram for uploading an image and connecting the custom controller

sequenceDiagram
    actor User
    participant WebUI as WebUI_master_control
    participant MaaFW
    participant MyCustomController
    participant Tasker
    participant AgentClient

    User->>WebUI: Select Custom tab
    User->>WebUI: Upload image file
    WebUI->>WebUI: on_upload(e)
    WebUI->>MaaFW: connect_custom_controller(e.content.read())
    activate MaaFW
    MaaFW->>MyCustomController: new(img_bytes)
    activate MyCustomController
    MyCustomController-->>MaaFW: instance with ndarray
    MaaFW->>MyCustomController: post_connection()
    MyCustomController-->>MaaFW: Future(wait).succeeded
    deactivate MyCustomController
    MaaFW-->>WebUI: True, None
    deactivate MaaFW
    WebUI-->>User: Show controller connected status

    User->>WebUI: Start task
    WebUI->>MaaFW: run_task(entry, pipeline_override)
    activate MaaFW
    MaaFW->>MaaFW: isinstance(controller, CustomController)
    MaaFW->>MaaFW: pipeline_override.update(entry -> DoNothing)
    MaaFW->>AgentClient: register_sink(resource, controller, tasker)
    AgentClient-->>MaaFW: bool
    MaaFW->>Tasker: post_task(entry, pipeline_override)
    Tasker-->>MaaFW: Future(wait).succeeded
    MaaFW-->>WebUI: succeeded, None
    deactivate MaaFW
    WebUI-->>User: Show task result without actions
Loading

Class diagram for MaaFW with MyCustomController and custom controller wiring

classDiagram
    class CustomController {
        <<abstract>>
        +post_connection() Future
        +post_click(x, y) Future
        +screencap() ndarray
        +connect() bool
        +request_uuid() str
    }

    class MyCustomController {
        +MyCustomController(img_bytes)
        +ndarray ndarray
        +connect() bool
        +request_uuid() str
        +screencap() ndarray
    }

    class AdbController {
        +post_connection() Future
        +post_click(x, y) Future
        +screencap() ndarray
        +connect() bool
        +request_uuid() str
    }

    class Win32Controller {
        +post_connection() Future
        +post_click(x, y) Future
        +screencap() ndarray
        +connect() bool
        +request_uuid() str
    }

    class Resource
    class Tasker {
        +post_task(entry, pipeline_override) Future
        +stop() void
    }

    class AgentClient {
        +register_sink(resource, controller, tasker) bool
    }

    class MaaFW {
        +resource Resource
        +controller AdbController
        +controller Win32Controller
        +controller CustomController
        +tasker Tasker
        +agent AgentClient
        +connect_custom_controller(img_bytes) Tuple~bool, str~
        +run_task(entry, pipeline_override) Tuple~bool, str~
        +click(x, y) bool
    }

    CustomController <|-- MyCustomController
    CustomController <|-- AdbController
    CustomController <|-- Win32Controller

    MaaFW --> Resource
    MaaFW --> CustomController
    MaaFW --> Tasker
    MaaFW --> AgentClient
Loading

File-Level Changes

Change Details Files
Introduce a CustomController-based implementation that serves frames from an uploaded image and wire it into MaaFW as a new controller type.
  • Import numpy and define MyCustomController that converts uploaded image bytes into a NumPy array and returns it from screencap.
  • Extend MaaFW.controller type annotation to include CustomController.
  • Add MaaFW.connect_custom_controller to instantiate MyCustomController with image bytes and perform post-connection initialization.
src/MaaDebugger/maafw/__init__.py
Adjust task execution and click handling behavior when using the custom controller to avoid performing real actions.
  • In MaaFW.run_task, detect CustomController and override the pipeline entry to a DoNothing action before posting the task.
  • In MaaFW.click, early-return False when the controller is a CustomController to prevent click dispatch.
src/MaaDebugger/maafw/__init__.py
Expose a new "Custom" controller option in the web UI that lets users upload an image to establish a custom controller connection.
  • Add a "Custom" tab in the controller connection UI and corresponding tab panel wiring.
  • Implement connect_custom_control that uploads an image, calls maafw.connect_custom_controller with the file content, updates GlobalStatus, and shows an error notification on failure.
  • Bind the custom controller tab into the existing controller_type storage alongside Adb and Win32.
src/MaaDebugger/webpage/index_page/master_control.py

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

你好——我已经审阅了你的修改,发现了一些需要处理的问题。

  • run_task 中,pipeline_override.update(...) 这行假设 pipeline_override 一定是一个字典;如果它可能为 None 或在调用方被省略,建议把它默认成 {},或者在调用 update 前做判断,以避免出现 AttributeError
  • run_task 里的两个分支都用完全相同的参数调用了 self.tasker.post_task(...).wait().succeeded;你可以先计算好被覆盖后的 pipeline,只调用一次,以减少重复代码。
  • connect_custom_control 中,maafw.connect_custom_controller 外层使用了裸 except,会吞掉所有错误(包括编程错误);建议只捕获预期的异常类型,并考虑记录错误详情,以便调试。
给 AI 代理的提示词
请根据这次代码审查中的评论进行修改:

## 总体评论
-`run_task` 中,`pipeline_override.update(...)` 这行假设 `pipeline_override` 一定是一个字典;如果它可能为 `None` 或在调用方被省略,建议把它默认成 `{}`,或者在调用 `update` 前做判断,以避免出现 `AttributeError`- `run_task` 里的两个分支都用完全相同的参数调用了 `self.tasker.post_task(...).wait().succeeded`;你可以先计算好被覆盖后的 pipeline,只调用一次,以减少重复代码。
-`connect_custom_control` 中,`maafw.connect_custom_controller` 外层使用了裸 `except`,会吞掉所有错误(包括编程错误);建议只捕获预期的异常类型,并考虑记录错误详情,以便调试。

## 单独评论

### 评论 1
<location> `src/MaaDebugger/maafw/__init__.py:179-180` </location>
<code_context>
-        return self.tasker.post_task(entry, pipeline_override).wait().succeeded, None
+        if isinstance(self.controller, CustomController):
+            # disable action
+            pipeline_override.update(
+                {entry: {"action": {"type": "DoNothing"}, "next": []}}
+            )
+            return (
</code_context>

<issue_to_address>
**issue:** 在调用 `update` 之前处理 `pipeline_override``None` 的情况。

在现有流程中,`pipeline_override` 可能为 `None`(之前是直接传给 `post_task` 的),但这个新分支假定它是字典,并对其调用 `update`,如果 `pipeline_override``None` 就会触发 `AttributeError`。建议在其为 `None` 时统一转换为一个空字典(例如 `pipeline_override = pipeline_override or {}`),或者让这个分支只在提供了字典时才执行。
</issue_to_address>

### 评论 2
<location> `src/MaaDebugger/maafw/__init__.py:111-115` </location>
<code_context>

         return True, None

+    def connect_custom_controller(self, img_bytes) -> Tuple[bool, Optional[str]]:
+        self.controller = MyCustomController(img_bytes)
+        self.controller.post_connection().wait()
+
+        return True, None
</code_context>

<issue_to_address>
**suggestion (bug_risk):** 检查 `post_connection()` 的结果,以与其他连接方法保持一致。

`connect_adb``connect_win32hwnd` 都会检查 `post_connection().wait().succeeded`,并在失败时返回 `(False, msg)`,但这个方法忽略了结果,总是返回 `(True, None)`。为了一致性及准确的错误报告,请在这里检查 `.succeeded`(以及可选的 `.message`),并在连接失败时返回失败的元组。

```suggestion
    def connect_custom_controller(self, img_bytes) -> Tuple[bool, Optional[str]]:
        self.controller = MyCustomController(img_bytes)
        result = self.controller.post_connection().wait()

        if not result.succeeded:
            # 在失败时重置 controller,以保持与其他连接方法一致的状态
            self.controller = None
            return False, getattr(result, "message", None)

        return True, None
```
</issue_to_address>

### 评论 3
<location> `src/MaaDebugger/webpage/index_page/master_control.py:310-316` </location>
<code_context>


+def connect_custom_control():
+    def on_upload(e):
+        GlobalStatus.ctrl_connecting = Status.RUNNING
+        try:
+            maafw.connect_custom_controller(e.content.read())
+        except:
+            GlobalStatus.ctrl_connecting = Status.FAILED
+            ui.notify("Failed to load image.", position="bottom-right", type="negative")
+            return
+
</code_context>

<issue_to_address>
**issue (bug_risk):** 处理上传错误时,避免使用裸 `except`。

裸 `except` 会隐藏所有错误,包括像 `TypeError` / `AttributeError` 这样的编程错误,并只显示一条通用提示。请只捕获你预期的上传/解码/连接相关异常,并考虑记录完整的异常信息,这样 `on_upload` 中的意外问题就不会被悄悄吞掉。
</issue_to_address>

### 评论 4
<location> `src/MaaDebugger/webpage/index_page/master_control.py:314` </location>
<code_context>
def connect_custom_control():
    def on_upload(e):
        GlobalStatus.ctrl_connecting = Status.RUNNING
        try:
            maafw.connect_custom_controller(e.content.read())
        except:
            GlobalStatus.ctrl_connecting = Status.FAILED
            ui.notify("Failed to load image.", position="bottom-right", type="negative")
            return

        GlobalStatus.ctrl_connecting = Status.SUCCEEDED

    StatusIndicator(GlobalStatus, "ctrl_connecting")
    ui.upload(auto_upload=True, on_upload=lambda e: on_upload(e))

</code_context>

<issue_to_address>
**suggestion (code-quality):** 使用 `except Exception:` 替代裸 `except:`[`do-not-use-bare-except`](https://docs.sourcery.ai/Reference/Default-Rules/suggestions/do-not-use-bare-except/)```suggestion
        except Exception:
```
</issue_to_address>

Sourcery 对开源项目免费——如果你觉得我们的审查有帮助,欢迎分享 ✨
帮我变得更有用!请对每条评论点 👍 或 👎,我会根据你的反馈改进后续的审查。
Original comment in English

Hey there - I've reviewed your changes and found some issues that need to be addressed.

  • In run_task, the pipeline_override.update(...) call assumes pipeline_override is a dict; if it can be None or omitted by callers, consider defaulting it to {} or guarding the update to avoid an AttributeError.
  • The two branches in run_task both call self.tasker.post_task(...).wait().succeeded with identical arguments; you can reduce duplication by computing the overridden pipeline once and then making a single call.
  • In connect_custom_control, the bare except around maafw.connect_custom_controller will swallow all errors (including programming errors); consider catching only the expected exceptions and optionally logging the error details to aid debugging.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `run_task`, the `pipeline_override.update(...)` call assumes `pipeline_override` is a dict; if it can be `None` or omitted by callers, consider defaulting it to `{}` or guarding the update to avoid an `AttributeError`.
- The two branches in `run_task` both call `self.tasker.post_task(...).wait().succeeded` with identical arguments; you can reduce duplication by computing the overridden pipeline once and then making a single call.
- In `connect_custom_control`, the bare `except` around `maafw.connect_custom_controller` will swallow all errors (including programming errors); consider catching only the expected exceptions and optionally logging the error details to aid debugging.

## Individual Comments

### Comment 1
<location> `src/MaaDebugger/maafw/__init__.py:179-180` </location>
<code_context>
-        return self.tasker.post_task(entry, pipeline_override).wait().succeeded, None
+        if isinstance(self.controller, CustomController):
+            # disable action
+            pipeline_override.update(
+                {entry: {"action": {"type": "DoNothing"}, "next": []}}
+            )
+            return (
</code_context>

<issue_to_address>
**issue:** Handle the case where `pipeline_override` is `None` before calling `update`.

In the existing flow `pipeline_override` can be `None` (it was passed directly to `post_task`), but this new branch assumes it’s a dict and calls `update`, which will raise `AttributeError` when `pipeline_override` is `None`. Consider normalizing to an empty dict when it’s `None` (e.g. `pipeline_override = pipeline_override or {}`) or guarding this branch so it only runs when a dict is provided.
</issue_to_address>

### Comment 2
<location> `src/MaaDebugger/maafw/__init__.py:111-115` </location>
<code_context>

         return True, None

+    def connect_custom_controller(self, img_bytes) -> Tuple[bool, Optional[str]]:
+        self.controller = MyCustomController(img_bytes)
+        self.controller.post_connection().wait()
+
+        return True, None
</code_context>

<issue_to_address>
**suggestion (bug_risk):** Check the result of `post_connection()` to align with other connect methods.

`connect_adb` and `connect_win32hwnd` both check `post_connection().wait().succeeded` and return `(False, msg)` on failure, but this method ignores the result and always returns `(True, None)`. For consistency and accurate error reporting, please check `.succeeded` (and optionally `.message`) here and return a failure tuple when the connection fails.

```suggestion
    def connect_custom_controller(self, img_bytes) -> Tuple[bool, Optional[str]]:
        self.controller = MyCustomController(img_bytes)
        result = self.controller.post_connection().wait()

        if not result.succeeded:
            # Reset controller on failure to keep state consistent with other connect methods
            self.controller = None
            return False, getattr(result, "message", None)

        return True, None
```
</issue_to_address>

### Comment 3
<location> `src/MaaDebugger/webpage/index_page/master_control.py:310-316` </location>
<code_context>


+def connect_custom_control():
+    def on_upload(e):
+        GlobalStatus.ctrl_connecting = Status.RUNNING
+        try:
+            maafw.connect_custom_controller(e.content.read())
+        except:
+            GlobalStatus.ctrl_connecting = Status.FAILED
+            ui.notify("Failed to load image.", position="bottom-right", type="negative")
+            return
+
</code_context>

<issue_to_address>
**issue (bug_risk):** Avoid a bare `except` when handling upload errors.

A bare `except` will hide all errors, including programmer mistakes like `TypeError`/`AttributeError`, and only show a generic notification. Please catch only the specific upload/decoding/connection exceptions you expect, and consider logging the full exception so unexpected issues in `on_upload` aren’t silently masked.
</issue_to_address>

### Comment 4
<location> `src/MaaDebugger/webpage/index_page/master_control.py:314` </location>
<code_context>
def connect_custom_control():
    def on_upload(e):
        GlobalStatus.ctrl_connecting = Status.RUNNING
        try:
            maafw.connect_custom_controller(e.content.read())
        except:
            GlobalStatus.ctrl_connecting = Status.FAILED
            ui.notify("Failed to load image.", position="bottom-right", type="negative")
            return

        GlobalStatus.ctrl_connecting = Status.SUCCEEDED

    StatusIndicator(GlobalStatus, "ctrl_connecting")
    ui.upload(auto_upload=True, on_upload=lambda e: on_upload(e))

</code_context>

<issue_to_address>
**suggestion (code-quality):** Use `except Exception:` rather than bare `except:` ([`do-not-use-bare-except`](https://docs.sourcery.ai/Reference/Default-Rules/suggestions/do-not-use-bare-except/))

```suggestion
        except Exception:
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copilot AI review requested due to automatic review settings December 4, 2025 12:00
@weinibuliu weinibuliu merged commit a523bfa into main Dec 4, 2025
7 checks passed
@weinibuliu weinibuliu deleted the feat/#141 branch December 4, 2025 12:01
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for uploading custom images as the target screen for image matching tasks, enabling users to test recognition pipelines against static images without needing a live controller connection.

Key Changes:

  • Introduces a MyCustomController class that implements CustomController to use uploaded images as screenshot sources
  • Adds a new "Custom" tab in the web UI that allows users to upload image files
  • Disables actions and clicks when using custom controllers to make tasks perform only image matching

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 11 comments.

File Description
src/MaaDebugger/webpage/reco_page/__init__.py Updates import path for cvmat_to_image to use the refactored img_tools module
src/MaaDebugger/webpage/index_page/master_control.py Adds "Custom" controller tab with file upload functionality for image-based matching
src/MaaDebugger/utils/img_tools.py Introduces rgb_to_bgr utility function for color channel conversion
src/MaaDebugger/maafw/__init__.py Implements MyCustomController class and connect_custom_controller method; modifies task execution to disable actions for static images

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

return False, f"Failed to connect {hex(_hwnd)}"

return True, None

Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The connect_custom_controller method is missing the @asyncify decorator that is present in other similar connection methods (connect_adb and connect_win32hwnd). This inconsistency could lead to issues if the method is called using await like the other connection methods in the UI code.

Suggested change
@asyncify

Copilot uses AI. Check for mistakes.

return True, None

def connect_custom_controller(self, img_bytes) -> Tuple[bool, Optional[str]]:
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter name img_bytes lacks type annotation. Other connection methods in this class consistently use type annotations for their parameters. Add : bytes type annotation for consistency.

Suggested change
def connect_custom_controller(self, img_bytes) -> Tuple[bool, Optional[str]]:
def connect_custom_controller(self, img_bytes: bytes) -> Tuple[bool, Optional[str]]:

Copilot uses AI. Check for mistakes.
Comment on lines +182 to +190
return (
self.tasker.post_task(entry, pipeline_override).wait().succeeded,
None,
)
else:
return (
self.tasker.post_task(entry, pipeline_override).wait().succeeded,
None,
)
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's code duplication in the if-else branches. Both branches return identical values (self.tasker.post_task(entry, pipeline_override).wait().succeeded, None). The logic can be simplified by updating pipeline_override conditionally before a single post_task call:

if isinstance(self.controller, CustomController):
    # disable action
    pipeline_override.update(
        {entry: {"action": {"type": "DoNothing"}, "next": []}}
    )
return (
    self.tasker.post_task(entry, pipeline_override).wait().succeeded,
    None,
)
Suggested change
return (
self.tasker.post_task(entry, pipeline_override).wait().succeeded,
None,
)
else:
return (
self.tasker.post_task(entry, pipeline_override).wait().succeeded,
None,
)
return (
self.tasker.post_task(entry, pipeline_override).wait().succeeded,
None,
)

Copilot uses AI. Check for mistakes.
GlobalStatus.ctrl_connecting = Status.SUCCEEDED

StatusIndicator(GlobalStatus, "ctrl_connecting")
ui.upload(auto_upload=True, on_upload=lambda e: on_upload(e))
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ui.upload component should specify accepted file types to prevent users from uploading non-image files. Consider adding the accept parameter with image MIME types, e.g., ui.upload(auto_upload=True, on_upload=lambda e: on_upload(e), accept="image/*").

Suggested change
ui.upload(auto_upload=True, on_upload=lambda e: on_upload(e))
ui.upload(auto_upload=True, on_upload=lambda e: on_upload(e), accept="image/*")

Copilot uses AI. Check for mistakes.
Comment on lines +316 to +318
ui.notify(
f"Failed to load image. {e}", position="bottom-right", type="negative"
)
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message is too generic. When image loading fails, users would benefit from more specific information about what went wrong (e.g., "unsupported image format", "corrupted image file", "image too large"). Consider catching specific exceptions (like PIL.UnidentifiedImageError) and providing tailored error messages.

Copilot uses AI. Check for mistakes.
Comment on lines +113 to +114
self.controller.post_connection().wait()

Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The connect_custom_controller method doesn't check the connection result like connect_adb and connect_win32hwnd do. Those methods verify post_connection().wait().succeeded and return failure status if unsuccessful. This method should similarly validate the connection and handle failure cases.

Suggested change
self.controller.post_connection().wait()
connected = self.controller.post_connection().wait().succeeded
if not connected:
return False, "Failed to connect custom controller"

Copilot uses AI. Check for mistakes.
Comment on lines +314 to +317
except Exception as e:
GlobalStatus.ctrl_connecting = Status.FAILED
ui.notify(
f"Failed to load image. {e}", position="bottom-right", type="negative"
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable name e shadows the upload event parameter from the outer scope. The exception should be given a different name to avoid confusion and improve code clarity. Consider using ex or err for the exception variable.

Suggested change
except Exception as e:
GlobalStatus.ctrl_connecting = Status.FAILED
ui.notify(
f"Failed to load image. {e}", position="bottom-right", type="negative"
except Exception as ex:
GlobalStatus.ctrl_connecting = Status.FAILED
ui.notify(
f"Failed to load image. {ex}", position="bottom-right", type="negative"

Copilot uses AI. Check for mistakes.
from ..utils.img_tools import cvmat_to_image, rgb_to_bgr


class MyCustomController(CustomController):
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The MyCustomController class name doesn't follow the naming convention of the codebase. Since it's implementing a custom controller for static images, a more descriptive name like StaticImageController or UploadedImageController would better convey its purpose and be more consistent with framework naming patterns.

Suggested change
class MyCustomController(CustomController):
class StaticImageController(CustomController):

Copilot uses AI. Check for mistakes.
Comment on lines +31 to +32
def request_uuid(self) -> str:
return "0"
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The request_uuid method returns a hardcoded "0" string. While this might be acceptable for a static image controller, consider documenting why this value is sufficient or using a more descriptive constant to clarify that UUIDs are not meaningful for static images.

Copilot uses AI. Check for mistakes.
return self.tasker.post_task(entry, pipeline_override).wait().succeeded, None
if isinstance(self.controller, CustomController):
# disable action
pipeline_override.update(
Copy link

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This expression mutates a default value.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants