[Benchmark] Guess correct input dtypes#230
Conversation
kozlov721
requested review from
conorsim,
klemen1999 and
tersekmatija
and removed request for
a team
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f8c91c9320
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
kozlov721
force-pushed
the
fix/incorrect-archives
branch
from
afcd8be to
c9b9961
Compare
| if model_precision_type == "FP32": | ||
| return DataType.FLOAT32 | ||
|
|
||
| return DataType.INT8 |
There was a problem hiding this comment.
Let's explicitly check for INT8 and if it doesn't match any of the cases explicitly error out. This is because shortly Hub will add mixed precision types and our backend will assume INT8 always with this code
| archive_precision: DataType | None, | ||
| hubai_precision: DataType | None = None, | ||
| ) -> DataType: | ||
| if dtype is DataType.INT32: |
There was a problem hiding this comment.
Where would this case apply?
| ) -> DataType: | ||
| if dtype is DataType.INT32: | ||
| return DataType.INT32 | ||
| if hubai_precision is not None: |
There was a problem hiding this comment.
Let's have some log when this happens so it is clearn from the CLI that precision was taken from HubAI "metadata". And similarly, let's log if precision was taken from the NNArchive metadata
| if shutil.which("snpe-dlc-info") is not None: | ||
| input_specs = self._get_dlc_input_specs(modelPath) | ||
| else: | ||
| model_archive = dai.NNArchive(modelPath) |
There was a problem hiding this comment.
I would still have this switched around so that we first try to read from DLC directly using SNPE tools and if that fails have the archive spec as a backup. Reason being that reading from DLC directly is 100% guranteed, other paths are not
2 participants
Purpose
Improved ability to guess the correct input types in case the information contained in the NN Archive is incorrect.
Specification
precisionfield from the HubAIGET /modelInstanceresponse is usually the most trustworthyDependencies & Potential Impact
None / not applicable
Deployment Plan
None / not applicable
Testing & Validation
None / not applicable
AI Usage
Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]
Submitted code was reviewed by a human: YES/NO
The author is taking the responsibility for the contribution: YES/NO