Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING) (Read 1507 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Hi all,

I'm releasing a new Monkey's Audio 11.00 with multi-threaded encoding and decoding.

Robert Kausch worked unbelievable magic again! He had baked this a while back and I prodded him a little and he helped get it totally ready to release!

Encoding or decoding a big file is many times faster. It uses the same concurrency settings previously, but uses more threads when doing less files than threads.

Here's a a build and SDK you could test:
https://www.monkeysaudio.com/files/MAC_1100.exe
https://www.monkeysaudio.com/files/MAC_1100_x64.exe
https://www.monkeysaudio.com/files/MAC_1100_SDK.zip

History:
NEW: Added multi-threaded encoding and decoding (thanks Robert Kausch!) (uses the same setting for concurrent files but when processing less files than threads it will increase the number of threads -- so one big CUE file will be much faster).
Changed: Bumped the interface version because the number of threads was added.
NEW: When compressing from the command line, specify something like -threads=16 after the compression level to set the number of threads. (example: MAC.exe "C:\1.wav" "C:\1.ape" -c2000 -threads=16 -t "Artist=Abba")
NEW: When decompressing from the command line, specify something like -threads=16 after the decompress flag to set the number of threads. (example: MAC.exe "C:\1.ape" "C:\1.wav" -d -threads=16")
NEW: When verifying from the command line, specify something like -threads=16 after the verify flag to set the number of threads. (example: MAC.exe "C:\1.ape" -v -threads=16")
Changed: Renewed the code signing certificate for another year.
Changed: Updated to Visual Studio 17.13.5.
Fixed: The installer made an extra entry in the uninstall section of the registry with the start menu folder but now it is with the other uninstall data.
Fixed: The path combobox did not allow entering a path longer than the combobox (it wasn't scrolling).
NEW: Added a thanks to Robert Kausch in the About box for all his genius (32-bit float, multi-threading, etc.).

Testing and any feedback appreciated.

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #1 –
Here are some times to decode a big CUE file on my computer:
Monkey's Audio 10.96: 9.94, 9.50
Monkey's Audio 11.00: 2.45, 2.52

So around 4x faster with my machine.


Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #2 –
Thanks for the new version. I did some tests. The results are good in my opinion.
However, when it comes to speed, HALAC produces really annoying results :o  The difference becomes much more dramatic, especially with larger files. The files in this test are of normal size.

Windows10 X64, Intel Core i5-2300
12 tracks, 44.1, 16 bit, stereo (585,144,122 bytes)
Quote
MAC 11.0 normal -threads=1 -> 394,231,260
Encoding 21.910 s, Decoding 22.489 s

MAC 11.0 normal -threads=8 -> 394,231,260
Encoding 6.439 s, Decoding 7.149 s
Quote
HALAC 0.3.8 -normal -> 404,597,742
Encoding 3.150 s, Decoding 4.239 s

HALAC 0.3.8 -normal -mt=8 -> 404,597,742
Encoding 1.758 s, Decoding 1.999 s

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #3 –
HALAC is a completely different thing. Let's go back to the early 2000s when FLAC and Monkey's were developed. Apart from the obvious common goal - to store PCM audio compressed - FLAC and Monkey's took very different priorities.

FLAC: designed for low decoding footprint, asymmetric in the sense that you can throw a lot of juice at the encoder (and initially you even had to, it wasn't that fast encoding), and what you got was a better predictor but essentially the same decoding.
Monkey's: designed for heavier compression, and symmetric. Think: an algorithm where decoding just does the encoding steps in reverse, better compression required a heavier algorithm for both.
Indeed, up to TAK we (at least I) thought it wasn't possible to get any such compression figures in a lightweight-decoding codec.

HALAC was designed to out-do FLAC at speed. And not only "FLAC": the fastest FLAC modes. You have a motorbike vs a truck, they both move from A to B but they serve different purposes. Don't complain that the bike takes less cargo and the truck is heavier.


And more. I might get some timeline wrong, but at least at some point in time:

FLAC: an audio compressor. Accepting WAVE and AIFF, but not getting you the file back, only the audio. Non-audio chunks support came much later and is only supported by the reference encoder.
Monkey's: a file compressor. Had a big picture in your WAVE file (not common then!)? Monkey's would store it, and you would get back not only the audio, but the same file bit by bit.
 
FLAC: out on Sourceforge, free(speech) software, free spec'ed format, enormous flexibility in what the encoded bitstream could look like.
Monkey's: full control over the bitstream, a Monkey's "high" file from 2006 is bit by bit the same as now up to the tags section.

FLAC: command-line tool for inclusion in other software
Monkey's: GUI, drag and drop and click convert

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #4 –
To be honest, I didn't like the truck and motorcycle analogy. As someone who has been working specifically with video codecs for years, I can easily say this. At least 95% of the total processing time in codecs is spent in stages such as prediction, filtering, hashing, entropy and encoding/decoding. Those who develop codecs know this very well.

These stages should already be present in HALAC. The remaining tiny time can be spent on other details. And a maximum of 5% speed loss required for these will not leave HALAC behind in any way. It tries to combine reasonable compression ratio with excellent speed balance. Maybe we will see more in the future. There hasn't been much movement or innovation in lossless audio for more than 20 years. At least we should thank HALAC for this.

I also now made tests with lossy audios produced with lossyWav (my favorite). However, Monkeys gave much worse results than the others. Even though I tried different parameters, the results did not change much.

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #5 –
Disclaimer: I don't use Monkey's for anything but testing, so my knowledge might be a bit off.


Quote from: genuine on
At least 95% of the total processing time in codecs is spent in stages such as prediction, filtering, hashing, entropy and encoding/decoding. Those who develop codecs know this very well.
I assume you have been working specifically with lossy codecs that are not suited for audio, but better data are known for lossless audio codecs. For example
FLAC profiling: https://hydrogenaud.io/index.php/topic,127409.msg1059257.html#msg1059257
The impact of audio checksumming: https://hydrogenaud.io/index.php/topic,123786.0.html
... and speed of fast verification: checksumming on the encoded stream: https://hydrogenaud.io/index.php/topic,125791.msg1042825.html (More formats do offer checksums on the encode and could do this fast verification - FLAC has block-level checksums, for example, but since FLAC is so fast decoding, the value added is much less than for slower codecs like Monkey's and OptimFROG.)

Quote from: genuine on
These stages should already be present in HALAC. The remaining tiny time can be spent on other details. And a maximum of 5% speed loss required for these will not leave HALAC behind in any way. It tries to combine reasonable compression ratio with excellent speed balance. Maybe we will see more in the future. There hasn't been much movement or innovation in lossless audio for more than 20 years. At least we should thank HALAC for this.
I agree. But it targets a different niche. FLAC does this, Monkey's does that. FLAC suits my needs better, so I use FLAC. And a little WavPack when I need it. Not HALAC, as it isn't mature, but some performance figures do impress me.

Quote from: genuine on
I also now made tests with lossy audios produced with lossyWav (my favorite). However, Monkeys gave much worse results than the others.
Of course it does, and it is very well known - see the "Hybrid/lossy" row of https://wiki.hydrogenaud.io/index.php?title=Lossless_comparison#Comparison_Table and https://wiki.hydrogenaud.io/index.php?title=LossyWAV#Codec_compatibility . 

But neither Monkey's nor FLAC were designed for lossy audio. Indeed, Josh Coalson listed "Lossy-compression" as one of the "Anti-goals" for FLAC.
Still, LossyWAV was (originally) designed for FLAC out of the idea to exploit a couple of properties of this format. Of course, that is one of the perks of FLOSS: once it is out in the free, you can exploit it even beyond what it was intended for.

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #6 –
But to stay on topic:

@MonkeysAudio : The 4x speedup you mention, is that with ... 16 threads, as in your example? And how many physical cores?
(Compare to FLAC figures in this thread: https://hydrogenaud.io/index.php/topic,124437.msg1040233.html#msg1040233 )

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #7 –
I have a i9-13900KF. I have Monkey's Audio set to eight files at once.

I tripped over a few problems with 11.00, so work pending.

If you decode to the end of the file then seek back and ask for more data it's no longer working.

Thanks for the help so far :)

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #8 –
I also added the FLAC encoding time. However, FLAC decoder does not support multithread.
Quote
MAC 11.0 normal -threads=1 -> 394,231,260
Encoding 21.910 s, Decoding 22.489 s

MAC 11.0 normal -threads=8 -> 394,231,260
Encoding 6.439 s, Decoding 7.149 s
Quote
FLAC 1.5.0 -5 -j1 --no-md5 -> 403,602,181
Encoding 7.738 s, Decoding 4.849 s

FLAC 1.5.0 -5 -j8 --no-md5 -> 403,602,181
Encoding 2.891 s, Decoding 4.849 s
Quote
HALAC 0.3.8 -normal -> 404,597,742
Encoding 3.150 s, Decoding 4.239 s

HALAC 0.3.8 -normal -mt=8 -> 404,597,742
Encoding 1.758 s, Decoding 1.999 s

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #9 –
I tested encoding a single WAV file running on a AMD Ryzen 9 5900x 12core 24threads:
2025-03-27  13:44       414,528,273 Fight - War Of Words - 1.5.0 10.55sec -8.flac
2025-03-27  14:03       414,820,798 Fight - War Of Words - 1096 08.59sec fast.ape
2025-03-27  14:02       407,605,266 Fight - War Of Words - 1096 11.25sec normal.ape
2025-03-27  14:01       406,039,058 Fight - War Of Words - 1096 11.96sec high.ape
2025-03-27  14:01       404,358,074 Fight - War Of Words - 1096 16.84sec extra high.ape
2025-03-27  13:59       403,409,178 Fight - War Of Words - 1096 33.95sec insane.ape
2025-03-27  13:53       414,820,798 Fight - War Of Words - 1100 01.65sec fast.ape
2025-03-27  13:52       407,605,266 Fight - War Of Words - 1100 02.32sec normal.ape
2025-03-27  13:51       406,039,058 Fight - War Of Words - 1100 02.42sec high.ape
2025-03-27  13:49       404,358,074 Fight - War Of Words - 1100 03.96sec extra high.ape
2025-03-27  13:51       403,409,178 Fight - War Of Words - 1100 08.32sec insane.ape
2025-03-27  13:44       628,767,542 Fight - War Of Words - source.wav

After that I tested encoding 44 wav files with all cores/threads available:
FFDP 1096 08.73sec extra high - 1238.84 MB.ape - LIMITED to 16 simultaneous files
FFDP 1100 09.45sec extra high - 1238.84 MB.ape - LIMITED to 16 simultaneous files
FFDP 1.5.0 09.89sec -8.flac - DbPowerAmp 24 threads
FFDP 1.5.0 10.02sec -8.flac - FooBar

In MonkeysAudio I guess 16 simultaneous files means 16 threads. Don't know why I cannot use all threads like I can in DbPowerAmp

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #10 –
I'll add this to the next release. Thanks for the help!

Changed: Expanded the allowed number of simultaneous files to 32.

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #11 –
Just uploaded again with a seeking fix and an increase to the number of simultaneous files possible. Thanks again for testing.

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #12 –
Quote from: MonkeysAudio on
Just uploaded again with a seeking fix and an increase to the number of simultaneous files possible. Thanks again for testing.
Encoding 44 wav files

Old: FFDP 1096 08.73sec extra high - 1238.84 MB.ape - LIMITED to 16 simultaneous files
Old: FFDP 1100a 09.45sec extra high - 1238.84 MB.ape - LIMITED to 16 simultaneous files
New: FFDP 1100b 08.14sec extra high - 1238.84 MB.ape - All 24 Threads I have.

Big gain again :-)
 

Re: Monkey's Audio 11.00 (MULTI-THREADED ENCODING / DECODING)

Reply #13 –
Thanks again for all the testing everyone.

I think I'll release tomorrow before leaving for the weekend unless anybody finds another problem.

Cheers :)