Conversation
1815fd9 to
cbabf51
Compare
|
Instance segmentation app is missing the inference timing. |
And model listing as well. It's because instance segmentation was merged after the PR with demo app fixes. I'll add these to instance segmentation. Good catch. DONE |
|
Additionally there are no inference timings for other speech models, but I understand it would be difficult to properly measure the inference time there as they operate in streaming mode. Maybe @IgorSwat has some idea how this could be done. |
|
@barhanc about streaming tasks, yeah I eventually decided that I don't have good idea how to present these times |
We could measure the average time of executing transcribe() inside the streaming loop - the question is whether we want to make a breaking change just for that. Same goes for Text to Speech - measuring it's inference time is possible, but would require digging into TypeScript API at very least. |
Co-authored-by: Bartosz Hanc <bartosz.hanc02@gmail.com>
Description
Added inference times, plus for some apps TTFT, tokens/second.
Fixed T2S streaming for unfinished sentences.
Introduces a breaking change?
Type of change
Tested on
Testing instructions
Screenshots
Related issues
Closes #959
Checklist
Additional notes