Skip to content

blender_agent.md

Latest commit

 

History

History
169 lines (111 loc) · 7.28 KB

blender_agent.md

File metadata and controls

169 lines (111 loc) · 7.28 KB

Blender Agent Infrastructure

1. Objective

Following is the backend workflow,

  1. The backend fastapi web server receives the request from the fronted client. Usually the client request consists of a text prompt, and optionally with one or multiple sketches or 2D images.

  2. The fastapi web server publishes the client request to the rabbit-mq message queue, and a lead agent subscribes to the rabbit-mq and receives the client request.

  3. The lead agent forwards the client request, via rabbit-mq, to a remote AI model Qwen2-72B-Chat to recognize the client intention.

  4. If the client request is to create 3D model based on the text prompt and attached sketches/images, the lead agent re-writes the prompt, and sends the modified prompt with the attached sketches/images to a sub-agent, via rabbit-mq, that uses a remote AI model Qwen3-VL-Plus to generate the detailed requirements in json format.

  5. When the lead agent receives the detailed requirements from the image recognition sub-agent, the lead agent writes a prompt based on the requirements, and sends the prompt to another sub-agent via rabbit-mq, and that sub-agent uses Qwen3-Turbo to generate Blender python script.

  6. The coding sub-agent sends the Blender python script back to the lead agent, and the lead agent sends the script via rabbit-mq to a Blender executor, the Blender executor uses asyncio.create_subprocess_exec() to spawn an isolated process to run the Blender 3D app in a headless/background mode. Once the job is finished, the Blender process is terminated.

  7. The Blender executor sends the generated .blend file, or .fbx or .gltf file, all the way back to the lead agent, to the fastapi, and finally to the client side. The user can download this file from his browser.

  8. The status of the job progress is stored in a postgre-sql database, the user can check the status of his job at any time.

 

2. RabbitMQ, PostgreSQL, and MinIO

2.1. Connections

A rabbit-mq message queue, a postgre-sql rdbms database, and MinIO object storage service, are deployed in a server.

Temporarily, they are deployed in a single node mode. In the future they will be deployed in a distributed way with multiple distributed nodes.

All agents, the Blender executor, and the Fastapi web server, can use the rabbit-mq, the postgre-sql, and the min-io across machines. That means for the convenience of development and testing, we can run the agents in our local macbook, and access the rabbit-mq, postgre-sql and min-io deployed in a remote server.

  1. When the agents, the blender executor, and the fastapi web server, send messages to each other, they use rabbit-mq.

    Every agent, the blender executor, and the fastapi web server, are micro-services. The purpose is to keep the entire backend server loosely decoupled.

    Each micro-service runs in an infinite loop, with access to the rabbit-mq.

  2. When the agents, the blender executor, and the fastapi web server, read or write data to the postgre-sql database, they connect the database directly via the database adapter SQLAlchemy.

    It is not necessary to wrap the postg. re-sql database into a micro-service and connect to the outside via rabbit-mq, because,

    • Overengineering: Adds a redundant layer (DB microservice) with no tangible benefits for our use case.

    • Latency: rabbit-mq + database microservice adds 50–200ms per database call (cumulative delay for multi-step workflows).

    • Consistency Risks: Async database updates can lead to race conditions (e.g., the lead agent reads "pending" status， while the sub-agent writes "completed").

    • Debugging Complexity: Database operations are hidden behind rabbit-mq messages (hard to trace "who updated job X").

  3. When the agents, the blender executor, and the fastapi web server, upload or download files to or from the min-io object storage service, they connect min-io via http/https, instead of using rabbit-mq.

    The reason is that rabbit-mq works better for small and frequent messages, but not large files.

  4. As mention aboved, when the agents, the blender executor, and the fastapi web server, send messages to each other, they use rabbit-mq.

    However, when they files to each other, they don't send the file via rabbit-mq. Instead, they take 2 steps.

    • Upload the file to the min-io,

    • Send the URL of the file to each other via rabbit-mq.

2.2. Rabbit-mq installation and running

The installation of rabbit-mq is not so easy as expected.

We successfully installed rabbit-mq in our ubuntu notebook, following rabbit-mq's official guidance.

  1. Install Essential Dependencies

    # Our ubuntu's version is 22.04, and its architecture is x86_64
$ uname -a
  Linux robot-test 6.8.0-87-generic #88~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Oct 14 14:03:14 UTC 2 x86_64 x86_64 x86_64 GNU/Linux


$ sudo apt-get update -y

$ sudo apt-get install curl gnupg -y

$ sudo apt-get install apt-transport-https


# Team RabbitMQ's signing key
$ curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null

  2. Create or modify /etc/apt/sources.list.d/rabbitmq.list.

    We changed arch=amd64 to arch=x86_64. But being honest, we don't know whether or not it is necessary.

    ## Modern Erlang/OTP releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-erlang/ubuntu/jammy jammy main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-erlang/ubuntu/jammy jammy main


## Provides modern RabbitMQ releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-server/ubuntu/jammy jammy main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-server/ubuntu/jammy jammy main

  3. Install erlang and rabbit-mq

    $ sudo apt-get update -y

# Install Erlang packages
$ sudo apt-get install -y erlang-base \
                        erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
                        erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
                        erlang-runtime-tools erlang-snmp erlang-ssl \
                        erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl

# Install rabbitmq-server and its dependencies
$ sudo apt-get install rabbitmq-server -y --fix-missing

  4. Run RabbitMQ Server

    # start it back
$ sudo systemctl start rabbitmq-server

# check on service status as observed by service manager
$ sudo systemctl status rabbitmq-server
  ● rabbitmq-server.service - RabbitMQ Messaging Server
     Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2025-12-15 17:04:53 CST; 38min ago

# Another useful status diagnostics tool
$ rabbitmq-diagnostics status

# stop the local node
$ sudo systemctl stop rabbitmq-server