Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: WavPack settings and buffer format (Read 7131 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

WavPack settings and buffer format

Hi.
I am using wavpack (as a library linked with my code) to compress 16 bit PCM data on a powerpc machine.
Since I get my buffers from another place already in memory, I don't use a real file, so wavpack actually reads the samples from a buffer in memory (i've implemented the needed callbacks for this).
My question:
1. I pass wavpack a buffer of int's. How should the samples be arranged inside the buffer?
Should I put each 16 bit sample in one int? Should it be aligned to the right or shifted to the left?
2. How should I configure wavpack? I use 16 bit for config->bits_per_sample, but what should i use for config->bytes_per_sample (and what does it mean)?
3. I tried using the following:
bits_per_sample=16
bytes_per_sample=4
Each int in the buffer was holding one 16 bit sample aligned to the right.
This yielded *bad* results (most of the buffers actually inflated).
4. When I am decompressing, wavpack writes data to a buffer. What is the format of the buffer. Is it 2 samples in each int or one sample aligned to the left/right?
5. One last thing, I am doing all this on a power pc cpu.

Again, I remind that I'm not letting wavpack read from a file but from a buffer in memory.

Thanks for any help,
Doron

WavPack settings and buffer format

Reply #1 –
Quote from: Doron on
Hi.
I am using wavpack (as a library linked with my code) to compress 16 bit PCM data on a powerpc machine.
Since I get my buffers from another place already in memory, I don't use a real file, so wavpack actually reads the samples from a buffer in memory (i've implemented the needed callbacks for this).
My question:
1. I pass wavpack a buffer of int's. How should the samples be arranged inside the buffer?
Should I put each 16 bit sample in one int? Should it be aligned to the right or shifted to the left?
2. How should I configure wavpack? I use 16 bit for config->bits_per_sample, but what should i use for config->bytes_per_sample (and what does it mean)?
3. I tried using the following:
bits_per_sample=16
bytes_per_sample=4
Each int in the buffer was holding one 16 bit sample aligned to the right.
This yielded *bad* results (most of the buffers actually inflated).
4. When I am decompressing, wavpack writes data to a buffer. What is the format of the buffer. Is it 2 samples in each int or one sample aligned to the left/right?
5. One last thing, I am doing all this on a power pc cpu.

Again, I remind that I'm not letting wavpack read from a file but from a buffer in memory.

Thanks for any help,
Doron


First, I assume you have read this. It covers this but I realize it's not super clear and I plan to update it soon.

For both packing and unpacking, WavPack always takes buffers of 32-bit signed integers, even if you're only passing in 8-bit or 16-bit values. This is done because WavPack always works in 32-bit internally. Assuming that the number of bits is a multiple of 8, then the values are right justified in the 32-bit integer. So, if your values are 16-bit ints, then you would simply cast them to a 32-bit int and pass them in (-32768 to +32767). Note that this means that the values must be signed-extended into the high bytes (I think this might be why your buffers were inflating). On decoding, the values are passed back exactly like they are passed in during encode.

For this example you would set bits_per_sample to 16 and bytes_per_sample to 2 (because the original samples fit in 2 bytes). The bytes_per_sample information is stored in the WavPack file so that during decoding the decoder will know how big the samples can be so that it can quickly detect errors (and it's also used for clipping in lossy mode).

There should not be any endian issue with the power pc. The samples are always passed in and out in the native endian mode of the processor, so you should not have to deal with this at all (unless you want to calculate md5 sums).

Hope this helps and let me know if you have more questions... 

WavPack settings and buffer format

Reply #2 –
Quote from: bryant on
First, I assume you have read this. It covers this but I realize it's not super clear and I plan to update it soon.

For both packing and unpacking, WavPack always takes buffers of 32-bit signed integers, even if you're only passing in 8-bit or 16-bit values. This is done because WavPack always works in 32-bit internally. Assuming that the number of bits is a multiple of 8, then the values are right justified in the 32-bit integer. So, if your values are 16-bit ints, then you would simply cast them to a 32-bit int and pass them in (-32768 to +32767). Note that this means that the values must be signed-extended into the high bytes (I think this might be why your buffers were inflating). On decoding, the values are passed back exactly like they are passed in during encode.

For this example you would set bits_per_sample to 16 and bytes_per_sample to 2 (because the original samples fit in 2 bytes). The bytes_per_sample information is stored in the WavPack file so that during decoding the decoder will know how big the samples can be so that it can quickly detect errors (and it's also used for clipping in lossy mode).

There should not be any endian issue with the power pc. The samples are always passed in and out in the native endian mode of the processor, so you should not have to deal with this at all (unless you want to calculate md5 sums).

Hope this helps and let me know if you have more questions... 


Thanks very much. This really helped. It appears that wavpack can't compress well my pcm samples because they were converted using a table from u-law samples. When I used an external application to convert the samples to linear pcm, wavpack handled the samples much better. Probably the external application does some kind of interpolation on the samples and this help wavpack compress. Anyway, I'll try now to activate the lossy compression and see the results.
The parameters for lossy compression are a bit vogue.
What's the difference between CONFIG_HIGH_FLAG, CONFIG_VERY_HIGH_FLAG and CONFIG_OPTIMIZE_WVC?
When I set a bitrate, will wavpack compress to this bitrate no matter what (on the cost of quality) or is this bitrate only a guideline? And one last question - what is weight shaping?

Thanks,
Doron

WavPack settings and buffer format

Reply #3 –
I think that if your original data is 8-bit u-law samples and you have access to that data, that would be the best source for WavPack. Converting to linear PCM should be worse. If you are seeing better performance with linear PCM than with the u-law, there might be a problem with the way the u-law is being passed to the encoder. This exact topic is actually discussed here:

http://www.hydrogenaudio.org/forums/index....showtopic=50743

There are 4 basic modes in WavPack characterized by the complexity (i.e. CPU demand) of the decoding. These are, in order of fastest decoding to slowest decoding, fast, normal, high, and very high. These are enabled by using one of these configure bits: CONFIG_FAST_FLAG, CONFIG_HIGH_FLAG or CONFIG_VERY_HIGH_FLAG. If you set none of these you get the default mode. On the command-line, the equivalent options are -f, -h, and -hh.

The configuration flag CONFIG_OPTIMIZE_WVC is equivalent to the -cc option on the command line and only applies to hybrid stereo. It may give somewhat greater compression, but since you're using u-law I'll guess you're not using stereo.

The bitrate you set is not a rigid limit, but it isn't exactly just a guideline either. I find with most music the bitrate varies between the value specified and about 10% over that. With some wacky material it might even go over that, but not often. If you need a specific bitrate that cannot be exceeded there's no easy answer right now, but I'm sure something could be worked out. In the worst case you could make another pass at the file/data with a new bitrate.

The noise shaping weight allows you to specify an alternate noise shaping parameter, equivalent to the -s command-line option. A value of 1.0 causes the quantization noise to be sloped upward at 6 dB/octave while a value of -1.0 gives -6 dB / octave. If your noise is actually audible you can vary this to make it less audible (or more pleasant).

David

WavPack settings and buffer format

Reply #4 –
Thanks a lot for your help!

Quote from: bryant on
I think that if your original data is 8-bit u-law samples and you have access to that data, that would be the best source for WavPack. Converting to linear PCM should be worse. If you are seeing better performance with linear PCM than with the u-law, there might be a problem with the way the u-law is being passed to the encoder. This exact topic is actually discussed here:
David


Well, I've tried keeping the u-law (normalizing it like the post said) but the performance are very bad - there's a lot of noise and crackles and it compresses a little less. The converted PCM samples on the other hand sound great and compress without a problem so I'll stick to that.

Anyway, now everything ticks like a clock.

Thanks for the help and the killer library!

Doron

WavPack settings and buffer format

Reply #5 –
Quote from: Doron on
Well, I've tried keeping the u-law (normalizing it like the post said) but the performance are very bad - there's a lot of noise and crackles and it compresses a little less. The converted PCM samples on the other hand sound great and compress without a problem so I'll stick to that.
If you're getting noise and crackles then I assume you're using lossy mode, which would not be appropriate for anything but linear PCM. If you're getting crackles with lossless then something is wrong. Of course, the u-law samples would still need to be converted back to PCM by something, so these WavPack files would not be playable by a regular WavPack app.

But yes, WavPack lossy is a far better lossy format than u-law, so if lossy is okay then that's perfect. If the u-law had to be preserved losslessly (i.e. no more degradation was allowed) then storing the u-law directly (before PCM conversion) would be better.

Glad it's working out for you. 

edit: added more about u-law and lossless