You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have several working QoL improvements and bugfixes that I'd like to upstream. Per the contributing guide, I wanted to open an issue first. Here's a list of several small, separate items that I'd like to submit a patch for:
Wheels built with the current pyproject.toml don't include any packages under src. The current dockerfile only works because it clones the entire source tree to /app and works out of that directory, so runtime imports of src.* packages import files from the source tree rather than the built wheel. Fix wheel builds not containing any src/ files #113
The config file must live at project_root / "openarc_config.json". This makes it somewhat difficult to place the config file on another volume, or change between config files. I have a patch that loads the config file path from an env var OPENARC_CONFIG_FILE, falling back to the current path if the var is unset. Get config file path from OPENARC_CONFIG_FILE if set #114
The config file is always written to upon startup, even when the existing config file contents are identical to the content to be written. This makes it impossible for the config file to exist on a read-only volume. This + the above hard-coded config file path limitation means that the application directory must be writable. I have a ~8 line patch that compares new config with the on-disk contents and short-circuits if they are identical. Avoid rewriting config file when nothing has changed #115
Models currently must be referenced in the config file via an absolute path, rather than a path relative to the config file. This makes writing config files ahead of time brittle, as the runtime path to the model must be hard-coded in the config file. I have a patch that allows loading models from a path relative to the config file allowing for config files like {"model_path": "./models-subdirectory/my-model"} instead of {"model_path": "/model/runtime/mount/path/models-subdirectory/my-model"}. Allow loading models from paths relative to config file #116
The Qwen3 ASR model re quires callers of the Whisper endpoint to always specify an openarc_asr field, which breaks the API contract and compatibility with other tools. I have a patch that falls back to default ASR config when this is request body field is unset, preserving the contract. Fix transcription endpoint requiring openarc_asr parameter when using Qwen3 ASR models #117
Compiled Qwen3 ASR models are not cached, causing them to be recompiled on every process restart, even on the same hardware and software stack. Compilation makes several copies of the model in memory, resulting in a huge increase in the peak memory required by the process; around 4GB above what is required post-model load. By simply setting the CACHE_DIR property on ov.Core() to a path specified by OPENARC_OV_CACHE_DIR, the peak memory requirement is reduced 3x after first startup. This allows for compiling the model once with these higher memory limits, then using lower memory limits for all future model load events (including across process restarts). Enable OpenVINO model caching #118
The server always attempts to write to app-directory/openarc.log. As with the config file saving issue, this requires that the application directory be writable, and writes logs to disk that users may not want. I have a patch that makes this configurable via env var, so that users can configure the server to write to /dev/null, /dev/std{out,err}, or another path of their choosing. Make file logging path configurable #119
After d7cb171 was merged, openarc serve start fails because the locked optimum version references a now-unavailable pytorch field and crashes. I have patch that bumps the lockfile to optimum 2. This works for my use case, but might need other people to test. Fix openarc serve start failing due to wrong dep version #120
Let me know if maintainers would accept some (hopefully all) of these patches. Most are just a couple of lines, with one or two reaching about a dozen lines. If it would be less toil for maintainers, I can combine these into a single ~100 line PR, or I can open separate PRs.
I have several working QoL improvements and bugfixes that I'd like to upstream. Per the contributing guide, I wanted to open an issue first. Here's a list of several small, separate items that I'd like to submit a patch for:
pyproject.tomldon't include any packages undersrc. The current dockerfile only works because it clones the entire source tree to/appand works out of that directory, so runtime imports ofsrc.*packages import files from the source tree rather than the built wheel. Fix wheel builds not containing anysrc/files #113project_root / "openarc_config.json". This makes it somewhat difficult to place the config file on another volume, or change between config files. I have a patch that loads the config file path from an env varOPENARC_CONFIG_FILE, falling back to the current path if the var is unset. Get config file path fromOPENARC_CONFIG_FILEif set #114{"model_path": "./models-subdirectory/my-model"}instead of{"model_path": "/model/runtime/mount/path/models-subdirectory/my-model"}. Allow loading models from paths relative to config file #116openarc_asrfield, which breaks the API contract and compatibility with other tools. I have a patch that falls back to default ASR config when this is request body field is unset, preserving the contract. Fix transcription endpoint requiringopenarc_asrparameter when using Qwen3 ASR models #117CACHE_DIRproperty onov.Core()to a path specified byOPENARC_OV_CACHE_DIR, the peak memory requirement is reduced 3x after first startup. This allows for compiling the model once with these higher memory limits, then using lower memory limits for all future model load events (including across process restarts). Enable OpenVINO model caching #118app-directory/openarc.log. As with the config file saving issue, this requires that the application directory be writable, and writes logs to disk that users may not want. I have a patch that makes this configurable via env var, so that users can configure the server to write to/dev/null,/dev/std{out,err}, or another path of their choosing. Make file logging path configurable #119openarc serve startfails because the locked optimum version references a now-unavailable pytorch field and crashes. I have patch that bumps the lockfile to optimum 2. This works for my use case, but might need other people to test. Fixopenarc serve startfailing due to wrong dep version #120Let me know if maintainers would accept some (hopefully all) of these patches. Most are just a couple of lines, with one or two reaching about a dozen lines. If it would be less toil for maintainers, I can combine these into a single ~100 line PR, or I can open separate PRs.