CVS-176786: Draft for Documentation update and deprecation notices #905
Conversation
…nfig parsing value error in documentation examples
|
|
||
| # Use config with session | ||
| options = {"device_type": "AUTO", "load_config": "ov_config.json"} | ||
| options = {"device_type": "AUTO", "load_config": json.dumps(config)} |
There was a problem hiding this comment.
Have you tested this change @Jaswanth51 ?
| ### `cache_dir` | ||
|
|
||
| **DEPRECATED:** This option is deprecated and can be set via `load_config` using the `CACHE_DIR` property. | ||
| **DEPRECATED:** This option is deprecated since OpenVINO 2025.3/ORT 1.23 and can be set via `load_config` using the `CACHE_DIR` property. `cache_dir` is configured **per-session** rather than globally. |
There was a problem hiding this comment.
what do you mean by "cache_dir is configured per-session rather than globally". Have you tested it yourself ?
There was a problem hiding this comment.
Does this part need to be mentioned? Aren't all properties set per session ?
| ```python | ||
| import onnxruntime as ort | ||
| import json | ||
| import openvino |
There was a problem hiding this comment.
what is the need to import openvino ? As far as ik, this is how it is used to wrok previously
if platform.system() == "Windows":
import onnxruntime.tools.add_openvino_win_libs as utils
utils.add_openvino_libs_to_path()
jatinwadhwa921
left a comment
jatinwadhwa921
left a comment
There was a problem hiding this comment.
Pls make the changes accordingly
MayureshV1
changed the title
MayureshV1
left a comment
MayureshV1
left a comment
There was a problem hiding this comment.
Can we also change line#90 and mention
The following provider options are deprecated since ORT 1.23 and should be migrated to load_config for better compatibility with future releases
| ### `cache_dir` | ||
|
|
||
| **DEPRECATED:** This option is deprecated and can be set via `load_config` using the `CACHE_DIR` property. | ||
| **DEPRECATED:** This option is deprecated since OpenVINO 2025.3/ORT 1.23 and can be set via `load_config` using the `CACHE_DIR` property. `cache_dir` is configured **per-session** rather than globally. |
There was a problem hiding this comment.
Does this part need to be mentioned? Aren't all properties set per session ?
| Enables model caching to significantly reduce subsequent load times. Supports CPU, NPU, and GPU devices with kernel caching on iGPU/dGPU. | ||
|
|
||
| **Benefits** | ||
| - Saves compiled models and `cl_cache` files for dynamic shapes |
There was a problem hiding this comment.
Can this be verified? GPU kernel vs model cache is based on a property- OPTIMIZE_SPEED, OPTIMIZE_SIZE.
-
- Particularly useful for complex models and frequent application restarts
This can be updated to: "Particularly useful for optimizing application startup latencies, especially for complex models"
- Particularly useful for complex models and frequent application restarts
| "DEVICE_PROPERTIES": { | ||
| "CPU": { | ||
| "INFERENCE_PRECISION_HINT": "f32", | ||
| "NUM_STREAMS": "3", |
There was a problem hiding this comment.
Please remove the INFERENCE_NUM_THREADS reference since it is a low level setting that is better ignored by app.
| "INFERENCE_NUM_THREADS": "8" | ||
| }, | ||
| "GPU": { | ||
| "INFERENCE_PRECISION_HINT": "f16", |
There was a problem hiding this comment.
For GPU you can show an example with using EXECUTION_MODE_HINT: ACCURACY in place of "INFERENCE_PRECISION_HINT".
Note, only 1 should be used since the execution mode internally will set the right inference precision hint.
| "AUTO": { | ||
| "PERFORMANCE_HINT": "THROUGHPUT", | ||
| "PERF_COUNT": "NO", | ||
| "DEVICE_PROPERTIES": "{CPU:{INFERENCE_PRECISION_HINT:f32,NUM_STREAMS:3},GPU:{INFERENCE_PRECISION_HINT:f32,NUM_STREAMS:5}}" |
There was a problem hiding this comment.
Nitpick: Can we be selective in the properties we show as example. We can reuse the ones we have used above.
in general using properties like INFERENCE_PRECISION or CPU would not yield much benefit since CPU only supports FP32.
Also, setting num_streams value to static would yield different result on each platform. it is better to use Performance_Hint which is portable across multiple platforms.
Added versions deprecation notices and updated documentation
https://jira.devtools.intel.com/browse/CVS-176786