Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Better Zip Compression (Read 42713 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Better Zip Compression

Reply #100 –
Another thing I thought of in this area...    is there any way to compress RAW-ISO data?  I know this is strongly tied to software piracy, but it was something I was always confused by.  We can compress execuatables, text, music, images to a great degree, but raw data can't be compressed.  WinRAR/ZIP rarely shave more than 1% off a cd-image of 800mb. 

Is this because nobody has attempted it, or because there is a fundamental difference that makes cd-images uncompressable? 

When I learnt of the failings of CD-R's in another thread, I thought of using CloneCD to take an image of the cd, storing it on my HDD.  Thing is, the image is the full size of the cd's, it can't be compressed in the same way as ripping the music to waves then using MAC.  ???
< w o g o n e . c o m / l o l >

Better Zip Compression

Reply #101 –
Quote
Mac wrote :

I can't find any actual links to a UHA page with the latest downloads, never got around to looking into it. Google will get you a french test between the latest archivers, (i put that link in so you can translate it). UHA came out very similar to 7Z smile.gif


here's the link to UHA binaries
Enjoy 
And maybe even post some good results 

Better Zip Compression

Reply #102 –
Quote
Another thing I thought of in this area...    is there any way to compress RAW-ISO data?  I know this is strongly tied to software piracy, but it was something I was always confused by.  We can compress execuatables, text, music, images to a great degree, but raw data can't be compressed.  WinRAR/ZIP rarely shave more than 1% off a cd-image of 800mb.  

Is this because nobody has attempted it, or because there is a fundamental difference that makes cd-images uncompressable?  

When I learnt of the failings of CD-R's in another thread, I thought of using CloneCD to take an image of the cd, storing it on my HDD.  Thing is, the image is the full size of the cd's, it can't be compressed in the same way as ripping the music to waves then using MAC.  ???

i just think it depends on the data on the cd.

if the data is compressable so will the image.
However you will offcouse not get mac compression on an iso audiotrack as
1: no generel compresser, compresses as well as mac on audio.
2: You have a lots of error checking data in between the audio destroying the "pattern" of the audio data (this goes for data tracks to)



but i have a nero image her which are compressed down to 59% of the original size.

--edit--
minor spelling errors
Sven Bent - Denmark

Better Zip Compression

Reply #103 –
madah,

Could you run some of those test suites with just a "-t7z -mx -r" on them?  I wonder how well a humane switch set would do instead of these suites that call for > 512 MB of RAM. 

As for CD image compression: Many CDs (especially software) already have compressed data on them, so trying to compress the image doesn't get you much.  Otherwise you'll need a compressor with a pretty large data dictionary to get good compression, because of the error checking bits and other stuff that appears in the datastream.

It would be interesting to see a specialized CD image compressor for handling things like keeping images on a PC and mounting as virtual drives.  The only possible problem would be the compression extracting a toll on the CPU.  Maybe with a good asymmetrical compression scheme, where decompressing is much less comlex than compression, the losses would be small enough that the overwhelming speed differential between CD and HD would win out.  <ponder ponder ponder>  Image mounting software would need some modification to handle this too...  Hmm...

Better Zip Compression

Reply #104 –
Quote
I can't find any actual links to a UHA page with the latest downloads, never got around to looking into it.

Here is a second link: http://dragoonhome.de/down/uharc04.zip

There you can download a GUI for UHARC (both in german & english).

Better Zip Compression

Reply #105 –
Quote
Could you run some of those test suites with just a "-t7z -mx -r" on them?


Roughly the same set of .xm-modules I've tested on earlier (461 MB uncompressed, but this time organised into several folders). Time measured on 450 MHz celeron:

Code: [Select]
7z -tzip -mx:   313 MB  23 min
rar3:           240 MB  15 min
ace2:           237 MB  34 min
7z -t7z -mx:    227 MB  38 min
7z (*):         194 MB  46 min
uharc (**):     160 MB  86 min


(*) = 7zn.exe a -t7z -r -y -m0=BCJ2 -m1=LZMA -m2=LZMA -m3=LZMA -mb0:1 -mb0s1:2 -mb0s2:3 -m1a=2 -m2a=2 -m3a=2 -m1d=16M -m2d=4M -m3d=4M -m1fb=128 -m2fb=128 -m3fb=128 archive.7z

(**) = uharc.exe a -m3 -md16384 -r archive.uha

WinRar 3 seems to be blazing fast, but it could be because of some problem with the timing...

Also there seems to be a problem when there's many files in many different folders; compared to the last test with exactly the same files:

zip: 2 MB larger
rar: 5 MB larger
7z: 17 MB (!) larger (was only 177 MB in the last test, compared to 194 MB in this one)

Edit: added uharc

Found a nice french site testing rar,7z,uharc: http://cmetge.dixinet.com/archive.asp

Better Zip Compression

Reply #106 –
Sometimes 7-zip maximum compression (no special atributes)  is better than with these atributes.

Better Zip Compression

Reply #107 –
Madah, I thought -mx was the best compression UHA had to offer? 

Thanks for the links to UHA, I searched some more before work, and came up with the same 0.4beta.  The GUI is handy.

If people are still interested - I can do some testing in a week when I get a new PC.  Everything on my current HD is compressed as it is to save space!!

I think UHA is at least a rival for 7Z, possibly better.


And on the CD Image business..  I hadn't thought about it for image mounting!  I remember I had a use for it once, but never had the gig's for more than a few images.  Maybe with compression it would be usable..

All I'm hearing though is "a cd image is raw data, you can't compress raw data"...  I can never find out why though!!
< w o g o n e . c o m / l o l >

Better Zip Compression

Reply #108 –
Quote
All I'm hearing though is "a cd image is raw data, you can't compress raw data"...   I can never find out why though!!

this arguments sound like BS.

if  the data has a repeatable pattern it can be compressed. its has nothing to due about it being "raw". data is data. only the pattern count about compressability.
Sven Bent - Denmark

Better Zip Compression

Reply #109 –
How does it explain winrar 3 taking an 800mb ISO and making it 800mb?  On maximum compression it does little or no different to on store mode (no compression) ?
< w o g o n e . c o m / l o l >

Better Zip Compression

Reply #110 –
Hey JuanK, thanks alot for that link. At last i have a UHA GUI .

Better Zip Compression

Reply #111 –
Quote
How does it explain winrar 3 taking an 800mb ISO and making it 800mb?  On maximum compression it does little or no different to on store mode (no compression) ?

you must be doing something worng or have you only tested with thise single is file ?

I just compressed a Clonecd image of 612mb down to 59% of original size with winrar3 (best/solid/4m)
the CloneCD was rip'ed with raw+96 reader. you just dont get more "RAW" then this (unlees you go do mdm which contains jitterinfo of the physical track)
I have taken a nero image down to 59% as well from 636mb with winace.

both these images are more "raw" then iso (iso is at a higher level)
Sven Bent - Denmark
 

Better Zip Compression

Reply #112 –
Quote
Madah, I thought -mx was the best compression UHA had to offer?


-mx uses the PPM mode which is mostly useful only for text-files (I think).

-m0 = store
-m1 = ALZ:fast
-m2 = ALZ:default
-m3 = ALZ:best
-mx = PPM
-mz = LZP

For best possible compression with uharc you can try:

uharc a -r -y -m3 -md32768 archive.uha

But lots of RAM is needed, otherwise uharc will use a lower amount of dictionary, like 16 or 8 MB.

Quote
I think UHA is at least a rival for 7Z, possibly better.


7z supports files and archives larger than 4 GB, filenames can be in unicode and it's open-source. Uharc has nothing of that...

Quote
How does it explain winrar 3 taking an 800mb ISO and making it 800mb? On maximum compression it does little or no different to on store mode (no compression) ?


This depends on the data. If there are installation files on the ISO they are certainly already packed with CAB or similar. What was the data you were trying to compress?

Better Zip Compression

Reply #113 –
Quote
-mx uses the PPM mode which is mostly useful only for text-files (I think).

Oh, I thought from the document that PPM was the best compression mode for anything.  I did a little test compressing a 2mb installer, PPM came out smaller than M3 (and than rar & 7z)..

Quote
uharc a -r -y -m3 -md32768 archive.uha

I was just using a simple uharc a -mx archive.uha!

How much ram would -md32768 need?  Are we talking the insane 512mb requirements I was seeing in some of 7Z's most hardcore modes?

Quote
7z supports files and archives larger than 4 GB, filenames can be in unicode and it's open-source. Uharc has nothing of that...

When I say better, I mean purely by compression ratio.  For day to day things, I'm going to stay with RAR, to me it is 'best' in terms of a simple GUI, multi-volumes, passwording, context menus, and better speed

Quote
you must be doing something worng or have you only tested with thise single is file ?

I just compressed a Clonecd image of 612mb down to 59% of original size with winrar3 (best/solid/4m)

Hehe, it could well be something done wrong.

I've tried it with an audio cd, games (compressed files), old games, which are runnable from the cd, and come with .wav music files!  It was a while ago, but I remember getting no compression from anything..  I'll try again and see if I can get results like you
< w o g o n e . c o m / l o l >

Better Zip Compression

Reply #114 –
Quote
Hey JuanK, thanks alot for that link. At last i have a UHA GUI .

You're welcome. 

Better Zip Compression

Reply #115 –
http://sjeng.org/ftp/7zboost.exe

I had a try at optimizing things, extract the above into your 7zip program dir.

I didn't benchmark. If you do, please post here.

Better Zip Compression

Reply #116 –
Garf,

When I try to use the 7zan.exe from your compile, I get a "libmmd.dll" not found.  7za.exe runs fine.  Where can I get that file at?

That said, I did a VERY rough test with GhostScript 7.04.  Clocked in at 1:10 for the default, and 0:54 for Garf's.

Better Zip Compression

Reply #117 –
Quote
When I try to use the 7zan.exe from your compile, I get a "libmmd.dll" not found.  7za.exe runs fine.  Where can I get that file at?

http://sjeng.org/ftp/libmmd.dll

It's from the compiler, I'm surprised it's needed.

Better Zip Compression

Reply #118 –
I've been doing some comparisons between RAR 3 and 7zip and usually 7zip wins out in compression ratio, but very slightly, and it seems (quite a bit?) slower than RAR.

The better-than-zip zipper is very nice though.

Better Zip Compression

Reply #119 –
BTW. Can anyone tell me how to configure Total Commander (EX-Windows Commander) to work with 7-zip?

Better Zip Compression

Reply #120 –
Quote
I had a try at optimizing things, extract the above into your 7zip program dir.

Maybe consider sharing Your otptimization ideas with author. Next beta will be faster

Better Zip Compression

Reply #121 –
Quote
http://sjeng.org/ftp/7zboost.exe

I had a try at optimizing things, extract the above into your 7zip program dir.

I didn't benchmark. If you do, please post here.

just curios but how did you optimize this
faster compiler ? mmx/sse/sse2/3dnow optimizing ?


--- edit ---

i just tried with you optimized files and the ddl file.
when compressin files or foldes by the context menu (rightclick) it say "specified module not found" nd closes it self instead of starting to compress files
Sven Bent - Denmark

Better Zip Compression

Reply #122 –
Quote
just curios but how did you optimize this
faster compiler ? mmx/sse/sse2/3dnow optimizing ?


--- edit ---

i just tried with you optimized files and the ddl file.
when compressin files or foldes by the context menu (rightclick) it say "specified module not found" nd closes it self instead of starting to compress files

Works fine here, did you download the second file also? (see thread above)

It's compiled with Intel C 6 + profile feedback. The source needed several modifications before it would compile in the first place, but most of those are farily trivial.

Better Zip Compression

Reply #124 –
Quote
Quote
just curios but how did you optimize this
faster compiler ? mmx/sse/sse2/3dnow optimizing ?


--- edit ---

i just tried with you optimized files and the ddl file.
when compressin files or foldes by the context menu (rightclick) it say "specified module not found" nd closes it self instead of starting to compress files

Works fine here, did you download the second file also? (see thread above)

It's compiled with Intel C 6 + profile feedback. The source needed several modifications before it would compile in the first place, but most of those are farily trivial.

yes i dowloaded "libmmd.dll" into the 7-zip  after i extracted the file from 7zip boost into the 7-zip folder.

my comp spec
dell latitude c810
p3 1.13ghz/733mhz tulatin cpu
Geforce2go 64mb
30gb toshiba disk
815 based motherboard
2x 256mb of pc133 ram cas3

-- edit --

finally got it to wrok be uninstalling and reinstalling a couple of times
Sven Bent - Denmark