Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: MuCodec - can compress audio down to just 0.35 kbps with impressive quality. (Read 1985 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

MuCodec - a neural, state-of-the-art, ultra-low bitrate music codec.
Can compress 48 kHz, stereo audio down to ONLY 0.35 kbps with impressive quality (for music, however)!  ;)
Link with demos and code: https://xuyaoxun.github.io/MuCodec_demo
Demos sound SO good, considering that it's compressed so SO HEAVILY. ~4390x!
Considering that a 24 kHz, stereo MP3 at 8 kbps sounds so underwater and xHE-AAC only tops down to 12 kbps for 44.1 or 48 kHz, stereo (by FhG's encoder), MuCodec compresses WAY MORE and SOUNDS BETTER. And it will blow SoundStream, EnCodec, Descript, FlowDec, SNAC, TSAC and other neural audio codecs out of the water!
Unfortunately, MuCodec isn't a all-around audio codec (meaning isn't trained on everything else and just trained on music) and sound effects and speech don't sound as good when compressed that extremely. But for music: this is very very efficient and very very good!
Also there is a 1.33 kbps mode for higher audio quality (especially on sound effects and other general audio) but this has a compression ratio of ~1155x!
If this is the future of audio codecs, I will replace MP3, xHE-AAC and all others with this codec.

EDIT: removed blank line as separator, also explained for xHE-AAC

Re: MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

Reply #1 –
I don't share your enthusiasm. While, as an experiment, it shows what technology can do, those samples sound horrible.
Yes, I get the bitrate, but what is the purpose of this codec? Low bitrate voice I understand, Earth <> Space communication, but I don't think that there is a space for these neural codecs, ESPECIALLY when they need so much processing power, memory, to even work.

I wont be listening music at such low bitrates.

These are pure experimental, proof-of-concept codecs.
TAPE LOADING ERROR

Re: MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

Reply #2 –
Quote from: itisljar on
I don't share your enthusiasm. While, as an experiment, it shows what technology can do, those samples sound horrible.
Music however sounds very good at that bitrate. I listened it and it sounds so good at that bitrate. However speech and general audio doesn't sound as good as music at that bitrate.
Also: storage is very very cheap today and today we have very fast internet. But they are still making codecs like this?? For what?
I get it, it's for very spotty and very bad network conditions (also with data caps in mind) and very limited storage conditions.
Also: fitting music clips in QR codes and weird, small storage locations with limited capacity.

EDIT: a new line of text added

Re: MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

Reply #3 –
This is practically the implementation of what I just had thought some months ago. Less than 2 kilobits per second can still represent billions of possibilities. Common audio codecs are more or less like intra-frame-only video coding, or with quite basic frame prediction. These AI audio codecs are like video coding with extreme frame prediction, lots of context, and complex transforms.

Though, the practicality of what we got with these proposals is still highly debatable. For me, it would be reasonable to have something that can compress audio down to 8 kbps with fair quality (xHE ~16k) and without having too high complexity.

Re: MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

Reply #4 –
Quote from: MihaiPopa12346 on
MuCodec - a neural, state-of-the-art, ultra-low bitrate music codec.
Yes, the internet is faster and storage is cheaper today than it used to be. And there is no need to deal with very low kbps audio. However, energy is still not cheap and there may be a dead end (AI, cryptocurrencies, etc.).
Codecs based on the kind of AI-powered brute-force learning model you are talking about require excessive energy and processing power. And processors are not getting faster anymore. We can only see increases in the number of cores. That's why fast solutions will always win.

https://www.in2013dollars.com/Electricity/price-inflation
https://fred.stlouisfed.org/series/PNRGINDEXM
https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Electricity_price_statistics

Re: MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

Reply #5 –
The problem lies in price of encoding and decoding. While encoding is done once, it still requires full AI engine and hardware which are expensive. Even then, decoding requires same engine and hardware to be usable. So, until hardware catches up or decoder is made which will run in (at least) real time on CPU, these codecs are just technology show, and nothing more.
Especially with sound like that. I don't know if we've listened to the same samples or not, but these are unusable for music listening. As an experiment, yeah, they sound good for the bitrate.
TAPE LOADING ERROR

Re: MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

Reply #6 –
Quote from: a.ok.in on
For me, it would be reasonable to have something that can compress audio down to 8 kbps with fair quality (xHE ~16k) and without having too high complexity.
Also it should include a 6 kbps mode with the quality of the xHE-AAC's 12 kbps mode.

Re: MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

Reply #7 –
Quote from: MihaiPopa12346 on
Quote from: a.ok.in on
For me, it would be reasonable to have something that can compress audio down to 8 kbps with fair quality (xHE ~16k) and without having too high complexity.
Also it should include a 6 kbps mode with the quality of the xHE-AAC's 12 kbps mode.
I understand. Unfortunately, you don't have the slightest idea about what we wrote.

Re: MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

Reply #8 –
Looks nice, but I don't know how to execute this at all.

Re: MuCodec - can compress audio down to just 0.35 kbps with impressive quality.

Reply #9 –
Quote from: Kamedo2 on
Looks nice, but I don't know how to execute this at all.
You need Python and various packages to run! :D
Also: I tried it (on Google Colab with a T4 GPU) and it doesn't work. Maybe someone more nerdy can fix the problem!