Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: The "business model" of hybrid codecs with correction files (Read 856 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

The "business model" of hybrid codecs with correction files

Hello HA,

It's been a while (about 10 years since I was last active). I see that there have been many improvements in the world of digital audio codecs, some of them quite impressive.

I just got a new Macbook Pro with 10 cores and it makes transcoding audio stupendously fast. I've been thinking for a while about moving my FLAC + Vorbis collection to either LossyFLAC or WavPack Lossy, + correction files. But the more I think about it, the less sense I can make out of it.

The problem with correction files is that they are "dead weight". On their own, you can't do anything with them. So, if you're going to back up a lossless collection, you can't just back up the correction files, you also have to back up the lossy files.

As it is now, I have only one copy of my Vorbis collection, on my 5 year old iPhone. If I lose it, no problem, I have my FLACs backed up, and those can be played back on their own (and, obviously, transcoded back to Vorbis or whatever).

With hybrid codecs, you end up with a single collection, which can be nice in case you need to modify tags or something. You do it once, and overwrite the modified lossy files on your device. The total size of a lossyFLAC / lossyWV collection with correction files, or a WavPack Hybrid collection, is somewhat smaller than FLAC + Vorbis. They are larger as a lossless collection than just FLACs (marginally).

I'm over 40 now and I probably couldn't ABX Opus at 128 kbps if my life depended on it. But I think we're just a couple of years away from having enough space on all our devices to just use FLAC and nothing else.

These are the problems I see with WavPack Hybrid, and LossyWAV:

  • WavPack Hybrid is slower and takes more space than FLAC. The lossy part is obviously larger than a pure lossy codec (about twice the size at 320 kbps than my ~160 kbps Vorbis files).
  • LossyWAV suffers from the same issue, whether you encode it with FLAC or WavPack (or TAK).
  • Correction files are supposed to live in the same directory as the lossy files, which makes it very inconvenient to copy an entire lossy collection to a DAP (say, my iPhone). I could write a BASH script (and I just might) to move lossy files and correction files to twin directory structures. That way I could just drag and drop the lossy directory structure to my iPhone. But I also would need to run that script in reverse to join lossy and correction files back together.
  • lossyWAV has a bigger problem of its own: the way it works is that you end up with two WAV files, that you encode with FLAC or whatever. So there's the time spent by lossyWAV to treat the original WAV, then the time spent on losslessly encoding TWO files instead of one. Then, when you want to losslessly transcode those, you need to decode both lossless files (so, twice the time), then use lossyWAV to merge the resulting two files (more time spent), and then finally transcode the reconstructed file to whatever you like. It's very slow, and very storage intensive (3 WAV files in total before you can transcode the 3rd file).

I think that purely lossy WavPack and lossyWAV are pretty cool. I guess some of you guys have such large music libraries that you may be tempted to archive it with a high quality lossy codec, suitable for transcoding, and abandon the idea of keeping a true lossless library.

But what do you guys think about correction files? Can anyone make sense of handling one's music collection that way?

Re: The "business model" of hybrid codecs with correction files

Reply #1 –
You can try an archive approach on the wvc correction files (zip / 7zip etc). You can relocated them to make things neater, then 'plug back in' to make lossless if required.

Re: The "business model" of hybrid codecs with correction files

Reply #2 –
The ability to create correction files in parallel during lossyWAV processing was more of an "it's possible" feature than an "it's space and time efficient" feature.

My music archive is FLAC and is converted to lossyFLAC for use on devices with limited storage capacity as and when required. I don't make use of the correction feature at all (apart from testing during development)

As a codec, rather than a DSP, WavPack has the significant advantage of being able to reconstitute lossy & correction files "on the fly".

Re: The "business model" of hybrid codecs with correction files

Reply #3 –
Quote from: skamp on
  • Correction files are supposed to live in the same directory as the lossy files, which makes it very inconvenient to copy an entire lossy collection to a DAP (say, my iPhone). I could write a BASH script (and I just might) to move lossy files and correction files to twin directory structures. That way I could just drag and drop the lossy directory structure to my iPhone. But I also would need to run that script in reverse to join lossy and correction files back together.
What about hardlinking? Create a "hardlinked mirror" of the entire collection, and then delete *.wvc from it afterwards.
Caveat: As far as I understand, the end-tags of .wv mean that retagging won't have to create a temporary file (thus breaking the hardlink structure) - but who knows if application X will do so?!

Also, you will have to use a player that actually plays lossless .wv+.wvc. foobar2000 for Windows does, so I'd be surprised if foobar2000 for MacOS cannot - but I think ffmpeg-based players will have to disregard the .wvc , limiting your choices.

Re: The "business model" of hybrid codecs with correction files

Reply #4 –
Quote from: skamp on
  • Correction files are supposed to live in the same directory as the lossy files, which makes it very inconvenient to copy an entire lossy collection to a DAP (say, my iPhone). I could write a BASH script (and I just might) to move lossy files and correction files to twin directory structures. That way I could just drag and drop the lossy directory structure to my iPhone. But I also would need to run that script in reverse to join lossy and correction files back together.
No need to do that (bold part).
You can just copy the whole hybrid music collection with all of its folder structure to another device with a filter that excludes *.wvc files, for example. There are file managers, file copying apps and scripts that let you do that easily. That is assuming the target device doesn't have some restrictions in terms of what software can access it.

And that's the nice thing about the hybrid approach. Having only one set of files to maintain and not needing any additional conversions, you just copy the lossy part over to any space-constrained device.

But, as you mentioned, the downside is that the lossy part of the hybrid codec (using WV here as an example) is just not very efficient. I didn't do much testing myself, but from what I've read, AAC achieves the same quality at half the bitrate. And it also decodes faster, for what it's worth.

Re: The "business model" of hybrid codecs with correction files

Reply #5 –
Quote from: Brand on
AAC [...] also decodes faster, for what it's worth.

I did a test on older Android phones: https://hydrogenaudio.org/index.php/topic,124857.0.html I did not test LossyFLAC nor WavPack Lossy though.
But apparently, doing FLAC by the OS' native decoding was lighter on battery. Suggesting that for battery life, one would choose a codec that is natively supported, AND a player application that would use that.
Also, avoid VLC.

No idea what happens on an iPhone, but if battery life is a concern then maybe one could test a couple of contenders.
 

Re: The "business model" of hybrid codecs with correction files

Reply #6 –
Quote from: Nick.C on
The ability to create correction files in parallel during lossyWAV processing was more of an "it's possible" feature than an "it's space and time efficient" feature.

Ah, the things we do just because we can 🙂

Quote from: Porcus on

What about hardlinking? Create a "hardlinked mirror" of the entire collection, and then delete *.wvc from it afterwards.
Caveat: As far as I understand, the end-tags of .wv mean that retagging won't have to create a temporary file (thus breaking the hardlink structure) - but who knows if application X will do so?!

That's a great idea. I totally forgot about hard links. I've been tinkering with your idea since I read your post and I didn't notice any issues with tagging a hard linked lossy .wv file (not with my own APEv2 tagger utility, anyway). No need to delete the correction files, I can just use a script to only hard link lossy files (and cover art JPEGs).

Quote from: Brand on
You can just copy the whole hybrid music collection with all of its folder structure to another device with a filter that excludes *.wvc files, for example. There are file managers, file copying apps and scripts that let you do that easily. That is assuming the target device doesn't have some restrictions in terms of what software can access it.

As much as I've been loving Apple products since my childhood, transferring new music to my iPhone in the foobar2000 (iOS) directory is a pain. The macOS interface only shows root directories of the foobar2000 data on my iPhone (in my case, artist directories) and you can only overwrite them as a whole, as far as I know. Possibly many subdirectories (album directories). The alternative is to transfer a new album directory to the iPhone's internal files storage, then manually moving that directory to the appropriate foobar2000 artist directory, using my iPhone. foobar2000 then takes care of re-scanning the collection.

I also don't know how to tell the macOS "Finder" to filter out certain filetypes while doing that.