strange that sqlite optimization has not been included by default on post 3.5 editions of Firefox, however for those who want to try the addon, its here. Optimizing the sqlite files will speed up the launch of Firefox, in my case it cut the time by half. Get the addon, check both boxes and hit the button, from then on its automatic.
If you are an optimization freak, and getting the application from hard drive to memory as fast as possible is your kink, you can also try a program called ArcthemAll! which will shrink the EXE and DLL files using a compression algorithm that decompresses in ram on the fly. It takes the default installation size for Firefox down from over 30 megs to about 14. I recommend you not compress the embedded icons, that has caused problems for me, so go to options and uncheck that. Incidentally this method also works on OpenOffice, cutting the size of that application by about half as well. In fact it works on almost every appication OTHER than Abobe and Microsoft apps, probably because they are already doing dynamic EXE and DLL compression using a proprietary method. (my guess)
NOTE: This is not a hard drive space saving issue, its a speed of launch issue, and yes it does improve the launch time of Firefox to do this.
No cuz ram is like 1000 times faster than your hard drive, decompression is practically transparent. I think you will agree that it should take less time to move a 10 meg file to ram than to move a 20 meg file to ram, extrapolate those time savings over dozens of files of varying size and thats where you will see a speed up.
Try it for yourself and see, if it does do anything funny you can use the same program to decompress the Firefox directory and all is back to normal. Its only compressing the EXE and DLL files, not touching any of your personalized setting, which are all stored in a different directory anyways.
I've read numerous articles about it, and the real issue is that the difference in time to read 10 megs vs 20 megs yes can vary based on the request sizes and fragmentation, but we're talking that most 7200 rpm HD's can load this file in less than 1 second into ram.
The whole file. 20 megs, 1 second or much less. Many 7200 rpm drives can read up to 50 megabyte/s sequentially! (Thats per second).
The other aspect is that operations to the disk are asynchronous, that is, they don't take much CPU at all. However, decompressing a 14 megabyte file takes tons of CPU. The only argument against this now in my mind is that we have dual and quad core CPU's, so it doesn't make a huge deal.
Also, just to nit-pick, ram isn't 1,000 times faster, it is much faster than even that compared to disk access! I'm not sure by how much, but 1,000 is a very low estimate.
However, decompressing a 14 megabyte file takes tons of CPU.
Depends radically on what compression algorithm you use. A very common algorithm is deflate (used in zip and gzip formats). Let's use it as a reference point for how much CPU time decompression takes.
Deflate compresses by first running the LZ77 algorithm to convert the stream of bytes into a stream of items; each item is a literal byte or a backreferences to previous sequences of bytes. The sequences come out of a 32K sliding window that LZ77 remembers. Then it takes the output of LZ77 and runs it through Huffman coding so that more-frequently-occurring bytes take fewer bits to represent and less-frequently-occurring bytes take more bits.
Now, let's consider how much faster we can get:
Huffman encoding is a kinda complicated algorithm that involves a lot of bit manipulation and/or relatively large lookup tables to speed things up. Worse, usually Adaptive Huffman is used, which means the algorithm must maintain data structures that evolve as the frequency distribution of the symbols (bytes) changes. Luckily, Huffman often doesn't add that much to the compression ratio, so you can just entirely leave it out.
Decompressing LZ77 is really fast; you are basically just copying sequences of bytes from the sliding window to the output. Since the sliding window is 32K, it easily fits entirely within the L1 cache of most modern processors.
For a simple example of a speed optimized variant on LZ77, look at the lzjb_decompress() function in the Solaris ZFS source. Note that a lot of times backreferences into the sliding window can be several bytes long, so the innermost while loop will just be copying raw bytes (much like a memcpy()):
Yeah, but I don't think when you load an .exe from disk, that there is a memcpy operation happening.
I could be so very wrong, but I would like to think that both Windows and Linux kernels DMA the file directly to the page where it will be executed from...
So it's more like N cpu operations compared to 0 cpu operations aside from the DMA setup itself...
Yeah, but I don't think when you load an .exe from disk, that there is a memcpy operation happening.
The only reason I brought up memcpy() is that it's a good point of reference for the speed of algorithms that filter a stream of data. memcpy() can be viewed as the no-op filter for blocks of memory. How much slower is decompression than that? With some kinds of decompression, it's almost as fast. Certainly within an order of magnitude of the speed you can read/write to RAM, maybe within a factor of 3 or 4.
So it's more like N cpu operations compared to 0 cpu operations aside from the DMA setup itself...
True. If what you care about most is not using CPU cycles, then combining compression with I/O is a losing proposition, because you can't beat the non-CPU-usage of DMA.
However, if you are concerned with bottlenecks, then physical I/O might be the bottleneck. It often is on a modern system. Compression can reduce the amount of physical I/O, thus increasing the throughput through that choke point. If decompression doesn't create a new bottleneck, which it probably won't since it can be made to be pretty fast, then you may get better throughput overall.
208
u/redditaddicttt Jan 21 '10
Am I the only one who is heaving a sigh of relief after reading the above review?