Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Time for a new lossless codec comparision? (Read 30340 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Time for a new lossless codec comparision?

Reply #25 –
Thks a lot for your test, I think it will affect my collection because I think I will switch from -6 to -4, it confirm what I was suspecting about-m & -M, -M is still interesting & removing it (like -3 or -0) will decrease the compression, but -m is just like -e ... a waste of encoding time for no gain (both -4 & -1 show it). This test made me realize that I was underestimating -M, in the switch I proposed to you I wanted to remove it which was not clever.

To be honest, it is not your test alone which will make me switch, it is the right time for me, I will soon have a new HDD & I need to fix offset on my collection with cuetools, but your test is a nice confirmation of my doubt about the usefullness of the -m switch which is used in -5/-6.

Here is how I analyse your flac result personnaly:


As for the way of displaying result, I think you can just use the same display as this test:
Hans Heijden's
It is pretty much perfect IMHO.

Just add encoders version on top, abbreviations of settings inside, also don't use the same color when result overlaps (tak & wv are green & mixed in your grap) & change the axis.

Time for a new lossless codec comparision?

Reply #26 –
Quote from: sauvage78 on
As for the way of displaying result, I think you can just use the same display as this test:
Hans Heijden's
It is pretty much perfect IMHO.

Just add encoders version on top, abbreviations of settings inside, also don't use the same color when result overlaps (tak & wv are green & mixed in your grap) & change the axis.

I'll try to do something similar
Quote from: C.R.Helmrich on
Agreed. Please label the axes. I assume the horizontal axis stands for speed and the vertical for compression ratio?
Also, speaking of ratios in percent, for better comparison, I suggest normalizing the speed axis to the results of the Shorten codec (since that's a classic reference, so to speak) and then to format it in percent as well, i.e. Shorten itself gets 100%, the fastest Flac mode gets nearly 200%, etc.

I was already busy finding out how to add labels, and I found out  Why should I scale to a reference codec instead of just plain wave? Now the axis inform you about how much a codec rougly compress, then the axis will inform you about how the codec outperforms Shorten... That's not really interesting, is it?
Quote from: Zarggg on
Quote from: ktf on
- Nine Inch Nails - The Slip (24-bit/96kHz)

Out of curiosity, why use this version of the track? Wasn't it shown previously that the 24/96 files were the same as 16/48 with padding?

Because it is the only 24-bit track I have... I didn't know it was fake. If it just padded, I won't use it  I have a 24-bit/96kHz capturing soundcard and a 24-bit/96kHz outputting guitar effect pedal, would that qualify for such a test?

edit: Moved

I added new music too. An about my harddrive: it is about 150 times realtime when copying from harddisk to ramdisk...
Music: sounds arranged such that they construct feelings.

Time for a new lossless codec comparision?

Reply #27 –
Quote from: ktf on
I was already busy finding out how to add labels, and I found out  Why should I scale to a reference codec instead of just plain wave? Now the axis inform you about how much a codec rougly compress, then the axis will inform you about how the codec outperforms Shorten... That's not really interesting, is it?


Because the information "times realtime" that you have now is a relative one, since it depends on your CPU. I was thinking of a way to make this an absolute measure. I agree that's not optimal either, so I guess we can stick with what you have. But then please provide your CPU brand and model, preferably in the axis label. That way I can estimate the speed for my system.

Chris


Edit: Now I get it, it seems you misunderstood me. I was only talking about the horizontal axis being put in relation to Shorten. The compression ratio (in relation to Wave) is fine the way it is
If I don't reply to your reply, it means I agree with you.
 

Time for a new lossless codec comparision?

Reply #28 –
Quote from: C.R.Helmrich on
Now I get it, it seems you misunderstood me.


Yes, I did  I was already thinking of making the 'times realtime' some more practical... My CPU is a AMD Turion ML-34, 1800MHz, 1MB cache, and runs on 64-bit Linux, but I don't think such information would be interesting to put in the graph, as MHz don't say anything this age. My idea was to run this test (partly) on more than one computer (I have three computers with quite different CPU's here) so I can look which universal benchmark would match.
Music: sounds arranged such that they construct feelings.

Time for a new lossless codec comparision?

Reply #29 –
Thanks, Turion ML-34 is already enough information

Quote from: ktf on
My idea was to run this test (partly) on more than one computer (I have three computers with quite different CPU's here) so I can look which universal benchmark would match.


Good idea. Do you have a slow(er) system available? Something in the range of an Atom, or a classic Athlon/Pentium III 1.x Ghz? Would be interesting, since Netbooks are gaining in popularity.

By the way, which albums/songs are included in your last posted analysis? I'm surprised that even the best-compressing codec doesn't reach a ration of 2/3! Has modern music really become that noisy?

Christian
If I don't reply to your reply, it means I agree with you.

Time for a new lossless codec comparision?

Reply #31 –
Quote from: C.R.Helmrich on
Do you have a slow(er) system available?


I have:
- This Turion ML-34
- A Server Pentium 3 of which I currently don't know the exact specs
- An Intel Celeron D, 64-bit

I (well, it is not mine, it is ours here) have a Celeron 400MHz, but it is being used actively, so I have to run this tests if they allow me

Edit: Removed, see http://www.icer.nl/losslesstest for the current results

As you can see, it is not balanced yet. Next I'll add some Jazz, and after that Classical. And yes, most metal (Mercenary in particular, you can listen some tracks here, see the albums 11 Dreams and The Hours That Remain) is very hard to compress. BTW, I added ALAC too. Don't forget to use F5, as I used the same filenames.

If someone can confirm that The Slip is indeed true 24-bit/96KHz (I downloaded it just a few weeks ago) I'll test these seperate
Music: sounds arranged such that they construct feelings.

Time for a new lossless codec comparision?

Reply #32 –
About NIN/slip I recall I re-downloaded it because there was a problem which was fixed, Trent even said thks you. I think it's OK now if I recall well.

See Here

Time for a new lossless codec comparision?

Reply #33 –
Updated the test again, visit http://www.icer.nl/losslesstest for the results. Please take a look at "What I'm going to do next..." and comment
Music: sounds arranged such that they construct feelings.

Time for a new lossless codec comparision?

Reply #34 –
I just finished the tests with instrumental material. I used the SQAM tracks, some tracks from Best Of Chesky Classics & Jazz and Audiophile Test Disc Volume 3 and some tracks from the Good bye Lenin Soundtrack and some tracks from Le Fabuleux Destin d'Amélie Poulain. In these tracks, there is just one instrument playing at a time (or someone speaking, or silence)

The results suprised me. LA doesn't outperform OptimFROG here, and some codecs do remarkable better (and some other remarkable worse) For example, see ALAC and the first 4 settings of FLAC... what a difference compared to the graphs at http://www.icer.nl/losslesstest! Also decoding shorten is much faster. Wavpack does quite bad in this test, probably because it needs filters (-x settings) to efficiently code this material? Something went wrong decoding the ape files. (that has to do with timing, I don't know what currently)





Music: sounds arranged such that they construct feelings.

Time for a new lossless codec comparision?

Reply #35 –
Maybe I didn't read the results closer enough but I noticed that there was only 7 points for flac when there are 9 flac settings, even considering that 5 & 6 are tied, there is one missing setting ( or a second tied ?). Plz put the settings & encoder versions abbreviation within the table, because it becomes unreadable without the numbers at hand: with a missing setting I don't know which point is which setting anymore. (Either flac -0 or -1 or -2 seems missing or tied from my understanding, results from -8 to -3 seems logic)

Time for a new lossless codec comparision?

Reply #36 –
Indeed, I see I forgot flac -0 for some strange reason... that's why these graphs were so different  I'll add them soon. (in about one hour I think)
Music: sounds arranged such that they construct feelings.

Time for a new lossless codec comparision?

Reply #37 –
Thanks a lot, ktf! Interesting results, and somewhat unexpected for me. So regarding your To-Do list: yes, I think you should keep plotting the instrumental results separate from the previous ones. The compression performance is very different.

Chris
If I don't reply to your reply, it means I agree with you.

Time for a new lossless codec comparision?

Reply #38 –
Updated. I also found the bug in my script which was causing the wrong values for APE, it's fixed now
Music: sounds arranged such that they construct feelings.

Time for a new lossless codec comparision?

Reply #39 –
Well gentlemen, indeed three years have passed since I last updated my lossless comparison page. As you read I'm still alive and well! Only my interests shifted a little.

Since I probably won't update the lossless page in the near future, thought some of you might be interested in the excel sheet behind the graphs. And also in an example of a batch file used to run a compress/decompress session. Here you can temporarily find a zip file with both of them.