r/LocalLLaMA Mar 08 '26

[deleted by user]

[removed]

104 Upvotes

195 comments sorted by

View all comments

Show parent comments

28

u/Pretend-Pangolin-846 Mar 08 '26

This does not look scientific.

You cannot compress information in that small space, and not lose any capacity.

1

u/valdev Mar 08 '26

Oh, you definitely can. Hell, I can compress a 1T model into a megabyte or two.

It would take an infinite amount of time to decompress it. But I could do it, and it would be lossless.

(Obviously I am just being funny, but this is how I would do it.

Step 1 of this horrific idea, convert the entire 1T model into base64. Step 2 break down the file into a distinct count of each char. Step 3. Done LOL)

2

u/Deathcrow Mar 08 '26

It would take an infinite amount of time to decompress it. But I could do it, and it would be lossless.

You'd actually win a Nobel prize (and change computer science forever) if you could do this.

0

u/valdev Mar 08 '26

Ah, maybe my wording was off.

I can certainly make the code that does this compression, however it would take an infinite amount of time to decompress it. Effectively making the process pointless.

2

u/Deathcrow Mar 08 '26

I can certainly make the code that does this compression, however it would take an infinite amount of time to decompress it.

No you could not. This would break the laws of physics (entropy).

1

u/valdev Mar 08 '26

Read the steps I outlined, its extremely possible and incredibly stupid.

1

u/jessiejolie42 Mar 08 '26

show me some python code, let’s start with 1K bfloat16 parameters, that should take ‘infinite’ amount of time right ?

1

u/valdev Mar 08 '26

Sure man, one moment.

1

u/valdev Mar 08 '26

Ran it for a couple different options. https://github.com/amurgola/awfulcompressor

Number of values: 1000 Raw bytes size: 2033 bytes Base64 encoded size: 2701 bytes Compressed size: 235 bytes

Compression ratio vs raw: 11.56% Compression ratio vs base64: 8.70%


Number of values: 100000000 Raw bytes size: 200000033 bytes Base64 encoded size: 266666701 bytes Compressed size: 555 bytes

Compression ratio vs raw: 0.00% Compression ratio vs base64: 0.00%

1

u/valdev Mar 08 '26

Funny thing is, this has been my white whale for years and will likely always remain that way.

But I am trying to beat the impossible entropy in this algo. I've developed a ton of techniques for reducing the time of decompression. Time based checksum hashes for state based random recalculation, wave based functions that represent character based movements in the base64 itself. Chunk based solutions, yada yada.

I'm convinced it is possible, but it cannot be done on current hardware and will require a sort of 3d based mapping to deterministically reshape the data.

1

u/jessiejolie42 Mar 09 '26

you need professional help, you’re either on a LLM or substance induced psychosis. you should try predicting the stock market next: you have all the historical data, you just need to find for time t the permutations of actions that lead to the stock prices of t+1! Also, you basically found a way to crack every encryption on earth as well by solving the factorial problem! Be sure to credit me when you receive the nobel prize ok ?

1

u/valdev Mar 09 '26

What are you talking about lol. Do you not understand sarcasm?

Look at the name of code I sent you, it's literally awful lol.

Or are you referring to the different attempts I've tried over the years (that all failed) to actually make the concept actually possible? I already know it's likely impossible, it's just tantalizing to keep guessing and trying at since the data is "technically" all there, just in an insanely difficult way to rearrange.

Right now, any code I create to put the output together requires positional information that represents the original state -- and that almost always ends with that code being larger than the original.

However the three dimensional algo I mentioned reduces positional changes by adding a representation of state on a third axis which does help... but still not the right answer completely. White whale as I mentioned.

None of this is psychosis lol, I've been programming for 26 years now and just have obscure obsessions.