LPCNet superseded by FARGAN

[jmvalin]

LPCNet is no longer being actively developed. It will continue to be available but for most applications, users are encouraged to switch to the Framewise Autoregressive GAN (FARGAN). FARGAN achieves better quality than LPCNet with just 600 MFLOPS complexity. That's 1/5 of the complexity of the most optimized LPCNet and 1/20 of the original LPCNet.

• J.-M. Valin, A. Mustafa, Jan Büthe, Very Low Complexity Speech Synthesis Using Framewise Autoregressive GAN (FARGAN) with Pitch Prediction, IEEE Signal Processing Letters, 2024, arXiv:2405.21069.

See our demo page for comparisons with LPCNet, HiFi-GAN, CARGAN and FWGAN. The PyTorch source code along with an optimized C implementation are available as part of the larger Opus codec implementation (FARGAN is used for PLC and deep redundancy within Opus).

[YYX666660]

Hi~
I've read the code and articles about FARGAN. In the latest opus 1.5.2 source code, FARGAN acts as a vocoder after the silk plc for improving the quality of synthesized speech, rather than directly for packet loss concealment.
I would like to ask if I want to train a specialized FARGAN network to perform packet loss concealment directly, what should I do? Is there a code repository implementation for this?
Thanks in advance,
Freya

[jmvalin]

FARGAN is a complete vocoder just like LPCNet. In Opus, the FARGAN signal completely replaces (not enhance) the SILK PLC output.

[YYX666660]

Thank you so much for your reply!

I'm a beginner of codec, and I'm not sure if my understanding is correct 🙏 In the Opus1.5.2 source code, when it comes to lpcnet_plc_conceal module, it seems that the process begins with using compute_plc_pred to update the features and states (using PLCModel, part of the LPCNet), and then compute_pitchdnn computes features for FARGAN (using PitchDNN model). Lastly, FARGAN (using FARGAN model) is used to synthesize pcm. So these three models are used to complete plc together?

If I want to perform plc directly, those three models must be trained and replaced the existing ones in Opus1.5.2, how can I train them separately? (only find the pytorch implementation of FARGAN)

[rafael2k]

Any simple-to-understand migration guide from those using LPCNet encoder / decoder for offline audio encoding and decoding?

[madscientist42]

1. Not everyone has the luxury of using PyTorch (especially in more advanced embedded systems...C'mon!)
2. Shouldn't you be providing that optimized C impl as a standalone like y'all did with LPCNet?

Since the above items are reality...you claim it, but it's not the same thing...not yet.

[jmvalin]

PyTorch replaces the TF code in LPCNet. Both still have a C implementation, so I'm failing to see what the issue is. There's also a standalone executable called fargan_demo which can be used in a similar was as lpcnet_demo.

[rafael2k]

Thanks @jmvalin. But as user of LPCNet, I'm in the same situation of @YYX666660, @madscientist42, and others. I also could not find the solution on how to use fargan the same way I use lpcnet.

Could you point us where to find the C code "fargan_demo" which could be used as drop-in replacement of "lpcnet_demo"?

ps: and we are all very excited to be able to use a true free software state-of-the-art ml-based audio encoder!

[jmvalin]

Like the README points out, all you need to do is build opus with --enable-deep-plc (or --enable-dred will work too) and it'll built a fargan_demo executable that you can use in a similar way to lpcnet_demo. The only difference is there's no longer a 1.6 kb/s compression mode, but that was never very good to begin with.

[rafael2k]

Thanks @jmvalin! Which bitrate fargan is using by default? For HF radio, 1.6 kbit/s was kind of in the limit of what is possible in a 3 kHz channel.

[jmvalin]

So there's two definitions that people use for a vocoder and a lot of people (myself included) will use them in a confusing way. FARGAN is a vocoder in the original meaning that it can generate speech from (uncoded) acoustic features. LPCNet started like that too, but gained a quantization mechanism at 1.6 kb/s in the same package. That quantization could technically be reused as-is for FARGAN, but I didn't bother because it's not super efficient.

That being said, if what you care about is speech over HF radio, then you might be interested in David Rowe's RADE (radio auto-encoder) work from FreeDV that uses FARGAN as a vocoder but has an encoder that directly generates the baseband signal. The resulting quality is much better than what 1.6 kb/s LPCNet could ever achieve (even without loss). You can read the paper at https://arxiv.org/pdf/2505.06671 and listen to some samples at https://freedv.org/davids-freedv-update-september-2024/

[rafael2k]

Hi @jmvalin. I'm well aware of RADE, but as you said, it also generates the baseband, which is not what one needs when integrating to any already existing modem. Also, as far as I know, it still has no optimized C implementation.

So just to check if I understood well - FARGAN has no quantization mechanism as LPCNet, so it is not a replacement of LPCNet for a super-low-bitrate audio encoder use case?

[snakers4]

Hm, I wonder why the paper does not mention vocos, which is a de-facto standard vocoder for neural TTS applications.

In a way it has superseded HiFi GAN, it is faster and easier to use and train, since it operates in spectral domain, without the need to compute heavy deconvolutions.

[jmvalin]

So just to check if I understood well - FARGAN has no quantization mechanism as LPCNet, so it is not a replacement of LPCNet for a super-low-bitrate audio encoder use case?

FARGAN is a replacement for the vocoder part of the LPCNet project, but it not a replacement for the compression part of it. It's likely that the compression code from LPCNet can just get ripped out and used for FARGAN, but I don't know if it's really worth it. The compression code was a quick and dirty proof of concept, but as I said I believe one could do a lot better today.

[jmvalin]

Hm, I wonder why the paper does not mention vocos, which is a de-facto standard vocoder for neural TTS applications.

Wasn't aware of vocos, so I had a quick look and I don't think it addresses the same needs as FARGAN. FARGAN is really aiming for small/efficient. As a comparison, the vocos paper lists models with 14M parameters (and no mention of weight quantization), whereas the largest FARGAN model has only about 800k parameters (~20x smaller) with all weights quantized to 8 bits.