Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Better Zip Compression (Read 42750 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Better Zip Compression

Reply #125 –
more compressiin benchmarks


33 folders and 396 .jpg's =  87,2mb

.rar = 87.1mb
.7z = 87.8

using
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=16M 2d=16M 3d=16M 1fb=64 2fb=64 3fb=64
This was as high as i could get with only 512mb of ram on my laptop

is still seems 7z is a no go for already compressed data. the size actually GREW. (please also se my last test with .aac files)
Sven Bent - Denmark

Better Zip Compression

Reply #126 –
...this is a bad configuration.
Try
Code: [Select]
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1d=25 2d=20 3d=20 1mf=bt4b 2mf=bt2 3mf=bt2

Better Zip Compression

Reply #127 –
Quote
...this is a bad configuration.
Try
Code: [Select]
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1d=25 2d=20 3d=20 1mf=bt4b 2mf=bt2 3mf=bt2

ok thanx.

I just stole the linie from Zombiek at page 4
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=16M 2d=16M 3d=16M 1fb=64 2fb=64 3fb=64


i question.
what the hell are all those parameters standing for ?

i kinda got the
1d 2d 3d 4d = dictionary size
1fb 2fb 3fb = fast bytes (whats is that)

whare are the rest

and why is there no FB in you line ?

I'm just DARN curios :-)


some more benchmarks


4282 files og 22 folders = 5.43gb
rar  = 1.12gb
7z  = 877mb

this is based on
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=16M 2d=16M 3d=16M 1fb=64 2fb=64 3fb=64

it shaved aditionalle 260mb of the file then winrar. Nice
Sven Bent - Denmark

Better Zip Compression

Reply #128 –
i just teste zombiek vs sk1's line

SK1 (0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1d=25 2d=20 3d=20 1mf=bt4b 2mf=bt2 3mf=bt2)
28.1mb


Zombiek (0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=16M 2d=16M 3d=16M 1fb=64 2fb=64 3fb=64)
72.mb


original size 91.2mb
Sven Bent - Denmark

Better Zip Compression

Reply #129 –
Quote
i just teste zombiek vs sk1's line


The reason why SK1's line was so much better could be that it uses 32M instead of 16M dictionary.
Can you try this line, if you have ~702 MB ram:

0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=64M 2d=4M 3d=4M 1fb=64 2fb=64 3fb=64

Quote
i question.
what the hell are all those parameters standing for ?


I suggest reading the 7-zip manual, section "Command Line Version / Switches / -m (Set Compression Method)"

1a=2 2a=2 3a=2
Means to use maximum compression for that stream:
a=[0|1|2]
Sets compressing mode: 0 = fast, 1 = normal, 2 = max ratio. Default value is 1.

1d=25 1d=32M
Means dictionary size, in this case both are 32M. 25 = 2^25 = 32M.
Default value for LZMA is 20 (1MB) in normal mode and 22 (4MB) in maiximum mode (-mx).
When BCJ2 is used dictionary sizes for methods 2 and 3 can be set to 1/8 of dictionary size of method 1.

mf={MF_ID}
Sets Match Finder for LZMA. Default method is bt4.
Memory requirements: d*9.5 + 6MB, where d = dictionary.
For the line at the top: 32*9.5+6 + 4*9.5+6 + 4*9.5+6 = 614 + 44 + 44 = 702 MB of memory.

0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3
I think this means which compression mode each stream will use and how they are connected. It is explained in the manual that BCJ2 has one input stream and four output streams.

1fb=64 2fb=64 3fb=64
Sets number of fast bytes for LZMA. It can be in range from 5 to 255. Default value is 32 for normal mode and 64 for maximum mode. Usually big number gives a little bit better compression ratio and slower compression process.


Edit: Quoted some from the 7-zip manual.

Better Zip Compression

Reply #130 –
Quote
Quote
i question.
what the hell are all those parameters standing for ?


I suggest reading the 7-zip manual, section "Command Line Version / Switches / -m (Set Compression Method)"

blah blah blah blah blah





thanx for the info. 


I'll chew on them later 




-- edit --
finnish chewing adn it helped a little

if
1a=2 2a=2 3a=2
Means to use maximum compression for that stream:
and
Default value is 1.

why are sk1''snot having them ? are they runnig with only 1 or by 2 (if u have chossed maximum in the gui)


-- edit --

BTW

does someone noe som switches that might decrease decoding time at coast of encoding time/ sligty bigget size

encoding time is not an importne issues for me
size and extracting time IS.
as I'm compiling driver cd's for mig works which alon uses 410mb of nvidia drivers.

i only have 650mb of space to use for all my drivers and test programs

sofar im using
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=32m 2d=1m 3d=1m 1mf=bt4b 2mf=bt2 3mf=bt2

if thise can ber improved for faster extraction please let me know
Sven Bent - Denmark

Better Zip Compression

Reply #131 –
Nobody else benchmarking UHA ?

I've done a small test... but it took my system so long I got bored of cleaning up my flat!


I might get accused of not giving 7Z a fair go by using the 20 switch command line, but I have 256mb of ram and so can't use a enourmous dictionary!


I think it's fair to say, out of the 3, they are pretty similar!!  UHA gave the best compression and least memory usage, WinRAR had the fastest compression & nicest features, 7Z had the fastest and least demanding decoding. 

  Personally, until a *BIG* drop in compression without loss of speed & frills I won't be switching from Rar



-=-=-=-=[my test]=-=-=-=-

Original Files :

277.1mb - Quake 2 / ICQ folders

1276 files / 97 folders  (exe/dll ~13% -=- zip/pak ~24% -=-  jpg/avi/mp3 ~8% -=- wav/tga/bmp ~28% -=- bsp/dm2 ~25%)



WINRAR 3.0 beta 2 (compression - best, solid, 4096kb dictionary, optimizations on auto or checked)

-Compression 124.3MB (44.9%)
-Time : ~9 min
-Memory : ~50 mb needed

-Decompression :  time ~1min 30
-Memory : ~30mb needed



7-ZIP 2.30 beta 4 (compression - maximum, solid)

-Compression 120.8MB (43.6%)
-Time : ~18 min
-Memory : ~50mb needed

-Decompression : ~1min 15
-Memory : ~8mb needed



UHARC  beta 0.4 (compression - ppm, solid, 4096kb dictionary)
   
-Compression 116.0MB (41.9%)
-Time : ~31 min
-Memory : ~25mb needed

-Decompression : ~15 min
-Memory : ~25mb needed
< w o g o n e . c o m / l o l >

Better Zip Compression

Reply #132 –
I'm using the following command "presets" for the different memmory usage/availbility)
there are all offspring of SK1 recomendet line but with adhustet dictionary size

(128 MB)
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=6m 2d=19 3d=19 1mf=bt4b 2mf=bt2 3mf=bt2

(256 MB)
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=16m 2d=1m 3d=1m 1mf=bt4b 2mf=bt2 3mf=bt2

(384 MB)
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=32m 2d=1m 3d=1m 1mf=bt4b 2mf=bt2 3mf=bt2

(512 MB)
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=48m 2d=2m 3d=2m 1mf=bt4b 2mf=bt2 3mf=bt2


But i think there might be wasted memmory on the sorting algoritme in the low memmory "presets", that might be better used for dictionary size.
Does someone have a better 256 or 128mb preset. or maybe a 64mb preset ?


maybe we should have a preset switch like the --alt-preset. Something like SK1=xxx wher xxx is the size of memmory usage :-D
Sven Bent - Denmark

Better Zip Compression

Reply #133 –
I just wanna say that 7-zip has slightly better compression and speed now:
"Some optimization recompiling for code size reducing."
"In max mode 7z now uses 8 MB dictionary instead of 4 MB."

Also:
"Self-extracting capability for 7z format."
AES encryption for files.

And more

Look there: http://www.7-zip.org/

Edit: If you want really best ZIP compression, use additional switches: fb=255 pass=4.

Better Zip Compression

Reply #134 –
it might be the best compression software, but it sure is slooooooooooooooooooooooooooooooooooow

Better Zip Compression

Reply #135 –
Quote
it might be the best compression software, but it sure is slooooooooooooooooooooooooooooooooooow

w4r3z kiddies don't care about compression speed, and that's where 7z rules 

Better Zip Compression

Reply #136 –
WinRar gives the use the ability to split the file after a given size.  Is this possible with 7z?

Better Zip Compression

Reply #137 –
Quote
WinRar gives the use the ability to split the file after a given size.  Is this possible with 7z?

This option will be supported soon...

Better Zip Compression

Reply #138 –
Quote
Quote
WinRar gives the use the ability to split the file after a given size.  Is this possible with 7z?

This option will be supported soon...

As soon as I find out that this option is supported, I will remove WinRar from my system and use 7z on its own!

Better Zip Compression

Reply #139 –
Funny - sometimes tar.7z is better than normal .7z

Better Zip Compression

Reply #140 –
Quote
Funny - sometimes tar.7z is better than normal .7z

7zip doesn't have native solid archive support?

Better Zip Compression

Reply #141 –
Quote
Quote
Funny - sometimes tar.7z is better than normal .7z

7zip doesn't have native solid archive support?

It does, which is what makes it funny.

I'm guessing file ordering may be the problem, i.e. sometimes tar is more lucky than 7z in ordering the files for good compression.

Better Zip Compression

Reply #142 –
is there any visual indication to how it's grouping them?  Does it do like WinRAR and group them by extension and then alphabetically?  Or just throw them all in the mix?

Better Zip Compression

Reply #143 –
Best known line for compressing Foobar2000:
7z.exe a -t7z foobar2000.7z -m0=BCJ2 -m1=LZMA -m2=LZMA -m3=LZMA -mb0:1 -mb0s1:2 -mb0s2:3 -m1a=2 -m2a=2 -m3a=2 -m1d=4M -m2d=4M -m3d=4M -m1fb=128 -m2fb=128 -m3fb=128 -mhcf=on -m1mf=bt4b fb2k_winnt\* fb2k_win9x\* extra\*

Best for packing PSI Jabber client:
0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=8M 2d=8M 3d=8M 1fb=128 2fb=128 3fb=128 hcf=on 1mf=pat3h with solid and "maximum"

Better Zip Compression

Reply #144 –
Lol!  W4r3z kiddies want compression rate, which is what 7-zip.org claims 7z gives

QUAKE2CD:\install\data\*.*
[span style='font-size:7pt;line-height:100%'](the instructions for installing q2 on win2000 is to copy this dir to your harddisk!)[/span]

Didn't measure speed or memory this time round, just pure size.  The 7-zip.org quote was annoying me after a friend was trying to convince me that 7z is the highest indeed the highest compression

RAR = 46.86%
7Z = 44.62%
UHA = 43.35%

As previously stated, UHA ()wnt j00!

However, as the smug friend pointed out...  this was a 388mb uncompressed archive, and the benefit of UHA over 7Z was 4.95mb 

And in the extra time it took to encode the 7z file over the rar file... I could of sent him 15mb, when 7z only saved 8.5mb  Swings and roundabouts!

I'm a smug w4r3z kiddy tho for now

[span style='font-size:7pt;line-height:100%']
RAR = max compression, solid, all special compression included
7z = 0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1d=25 2d=20 3d=20 1mf=bt4b 2mf=bt2 3mf=bt2 (SK1's line)
UHA = PPM 4mb dictionary
[/span]
< w o g o n e . c o m / l o l >

Better Zip Compression

Reply #145 –
Quote
[span style='font-size:7pt;line-height:100%']
RAR = max compression, solid, all special compression included
7z = 0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1d=25 2d=20 3d=20 1mf=bt4b 2mf=bt2 3mf=bt2 (SK1's line)
UHA = PPM 4mb dictionary
[/span]

Try 0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=16M 2d=16M 3d=16M 1fb=128 2fb=128 3fb=128 hcf=on 1mf=bt4b  (!)

Better Zip Compression

Reply #146 –
Quote
Try 0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=16M 2d=16M 3d=16M 1fb=128 2fb=128 3fb=128 hcf=on 1mf=bt4b  (!)

IIRC the author of 7-zip said it's not necessary to make all d-Values (1d, 2d, 3d) equal. Instead of
1d=16M 2d=16M 3d=16M it should be better to use something like
1d=32M 2d=2M 3d=2M (or 32M/1M/1M as in SK1's CL)

Have you tried this?
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

Better Zip Compression

Reply #147 –
Quote
Quote
Try 0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=16M 2d=16M 3d=16M 1fb=128 2fb=128 3fb=128 hcf=on 1mf=bt4b  (!)

IIRC the author of 7-zip said it's not necessary to make all d-Values (1d, 2d, 3d) equal. Instead of
1d=16M 2d=16M 3d=16M it should be better to use something like
1d=32M 2d=2M 3d=2M (or 32M/1M/1M as in SK1's CL)

Have you tried this?

16 8 8 for FB2K archive was WORSE than 4 4 4 

Better Zip Compression

Reply #148 –
Quote
Try 0=BCJ2 1=LZMA 2=LZMA 3=LZMA b0:1 b0s1:2 b0s2:3 1a=2 2a=2 3a=2 1d=16M 2d=16M 3d=16M 1fb=128 2fb=128 3fb=128 hcf=on 1mf=bt4b  (!)

  That line ends up bigger by 0.8mb...

174,358,723 instead of 173,552,904

Once my friend has recieved the archive, I'm sure he won't sleep until he's found a line that beats UHA
< w o g o n e . c o m / l o l >

Better Zip Compression

Reply #149 –
Ok you tried compressing one set of files with 7-zip and UHA. Have you done any others? Before you get all smug perhaps you should try more file sets. There are sets that RAR beats UHA and 7-zip. Likewise there are sets that 7-zip beats RAR and UHA. And then of course there are sets that UHA beats RAR and 7-zip. But quite often it seems 7-zip does a bit better than others. But not always mind you.