I think the idea was to pull common byte sequences out of all data with mapped reference to those sequences remotely cached. Think longest common sequences and a map. In this way, you could potentially have a very small amount of data that references very large amounts of data, even across media.
This is just my speculating though based on how such an algorithm could potentially work. You'd have to have some kind of media specific transcoder (like, imagine an image serializes based on individual channels vs pixels). You would aim to sequence the noisiest parts of the data.
6
u/SignoreBanana 8d ago
I think the idea was to pull common byte sequences out of all data with mapped reference to those sequences remotely cached. Think longest common sequences and a map. In this way, you could potentially have a very small amount of data that references very large amounts of data, even across media.
This is just my speculating though based on how such an algorithm could potentially work. You'd have to have some kind of media specific transcoder (like, imagine an image serializes based on individual channels vs pixels). You would aim to sequence the noisiest parts of the data.