What is the best version?

[bukhalmae145]

There are variety of version in Grad SVC (V3 CFM, V3 CFM RoPE, V2 96, etc..), but what is the best version?

[MaxMax2016]

v2 96 with Integrated Fast Maximum Likelihood Sampling Scheme.I have not test so much. maybe CFM's advantage is speed.

[bukhalmae145]

v2 96 with Integrated Fast Maximum Likelihood Sampling Scheme.I have not test so much. maybe CFM's advantage is speed.

So you basically mean quality=v2 96 and speed=v3 CFM?

[MaxMax2016]

yes, CFM can use less steps.

[bukhalmae145]

yes, CFM can use less steps.

So Grad-SVC is better than so-vits-svc 5.0 and the best version of Grad-SVC is v2 96, right? What is the difference between Grad-SVC v2 and v3?

[MaxMax2016]

Different parameters have different effects, such as big model gets better result. Grad-SVC & so-vits-svc 5.0 are all just demo with small model for SVC, Their true abilities have not been tested.

Grad-SVC v2 uses Fast Maximum Likelihood Sampling Scheme from https://github.com/huawei-noah/Speech-Backbones/tree/main/DiffVC

Grad-SVC v3 uses CMF from https://github.com/shivammehta25/Matcha-TTS

[bukhalmae145]

Different parameters have different effects, such as big model gets better result. Grad-SVC & so-vits-svc 5.0 are all just demo with small model for SVC, Their true abilities have not been tested.

Grad-SVC v2 uses Fast Maximum Likelihood Sampling Scheme from https://github.com/huawei-noah/Speech-Backbones/tree/main/DiffVC

Grad-SVC v3 uses CMF from https://github.com/shivammehta25/Matcha-TTS

Do you have any plan to release or develop the best SVC model?

[MaxMax2016]

https://www.zhangxueyao.com/data/MultipleContentsSVC/index.html some one else may will do.

[bukhalmae145]

https://www.zhangxueyao.com/data/MultipleContentsSVC/index.html some one else may will do.

May I know how gvc.pretrained.pth is built?

[MaxMax2016]

train by the open source data：https://github.com/Multi-Singer/Multi-Singer.github.io

[markrmiller]

I'm having a real hard time matching your loss graph results.

I've tried fine tuning a single voice with about 1400 3-10 second samples. I've also tried fine tuning on 5 such voices at once. In both cases, the results continue to get better, but it took thousands of epochs and millions of iterations to get decent and the loss results are still not down to where you end up in the published charts. Bumping up the learning rate didn't appear to help much.

I've also tried training from scratch using the above dataset. At about 900,000 iterations, it's nowhere near what I see in your loss charts. The encoder melspectrograms are still more noise than melspectrogram. The other loses have also hardly come down.

Any thoughts?

[markrmiller]

Just to answer my own comment: it was normal not to see loss graphs like the ones I was looking at in the repo. The 3 main losses are not typical. Diffusion loss will generally quickly drop initially, but then can spend a very long time in the same range as every time step gets trained. Then the other three losses are adversarial, and so somewhat similar, though prior and Mel will slowly drop over time.

I played around a lot, but in the end I replaced the attention in the encoder with flash attention 2, swapped out the continuous diffusion model with diffusers, with a much larger unet and their schedulers and trained on 4k voices (about 500k+ clips) Also moved it to latent space with a VAE. Trains many, many times faster and can do inference in like 1-3 steps. Great project to play with, thanks!

[bukhalmae145]

Just to answer my own comment: it was normal not to see loss graphs like the ones I was looking at in the repo. The 3 main losses are not typical. Diffusion loss will generally quickly drop initially, but then can spend a very long time in the same range as every time step gets trained. Then the other three losses are adversarial, and so somewhat similar, though prior and Mel will slowly drop over time.

I played around a lot, but in the end I replaced the attention in the encoder with flash attention 2, swapped out the continuous diffusion model with diffusers, with a much larger unet and their schedulers and trained on 4k voices (about 500k+ clips) Also moved it to latent space with a VAE. Trains many, many times faster and can do inference in like 1-3 steps. Great project to play with, thanks!

Does the version you modified have a better quality? If it is, can you share the source code please?