Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Has accelerating crc with avx512 been investigated? (Read 1057 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Has accelerating crc with avx512 been investigated?

This ( https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=3c6e8c123896584f1be1fe69aaf68dcb5eb094d5 ) update to postgresql caught my eye, where they improved the speed of their crc32c implementation (relative to using the intel crc32c instructions https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#othertechs=CRC32 ) by using avx512 instructions to process 64 bytes at a time. If you drill down they derived their implementation from here https://github.com/corsix/fast-crc32/ . I know it's a different type of crc to the crc8 and crc16 used in flac, but the technique may be applicable. Postgresql gained a good speedup relative to native instructions, we may see an even bigger gain as our implementation is table-based which should be slower than a native instruction would be.

crc isn't a massive proportion of runtime in libflac, but it's not insignificant either and is calculated on both encode and decode: https://hydrogenaud.io/index.php/topic,125248.msg1052232.html#msg1052232

If it hasn't been investigated already and if an implementation of it might be accepted in libflac, I'll investigate further time permitting.

Re: Has accelerating crc with avx512 been investigated?

Reply #2 –
I have no AVX512 capable CPUs so I cannot test any improvements specific to it. That makes it difficult to merge any such improvements.

Edit: also, the decoder currently doesnt use intrinsics. I'd like to keep it that way except for when a large improvement comes with it.
Music: sounds arranged such that they construct feelings.

Re: Has accelerating crc with avx512 been investigated?

Reply #3 –
If the decoder isn't going to get any such intrinsincs, then a potenitally semi-interesting use could be a standalone fast-verifier that does not decode.

Maybe interesting because: offical FLAC does not have anything such. OptimFROG, Monkeys and WavPack can ( https://hydrogenaud.io/index.php/topic,125791.msg1042825.html#msg1042825 , watch out for corrections to my blunt blunders), so they verify faster than FLAC.

Maybe not-so-interesting because: FLAC decodes so fast that who cares.
(Enthusiasts do.)
 

Re: Has accelerating crc with avx512 been investigated?

Reply #4 –
Quote from: ktf on
I have no AVX512 capable CPUs so I cannot test any improvements specific to it. That makes it difficult to merge any such improvements.

Edit: also, the decoder currently doesnt use intrinsics. I'd like to keep it that way except for when a large improvement comes with it.
Understandable. Might give it a go anyway as a proof of concept, wrapping my head around the maths of it would be a good exercise. BTW avx512 is fun if you like that sort of thing, and now intel have backtracked on the whole avx10.256 thing so the path forwards is much less of a mess.