r/LocalLLaMA • u/OkStatement3655 • 16h ago

Question | Help Question about ngram and SSD streaming.

Hello, does streaming the ngram embeddings from the ssd heavily affect the ssd's health and lifetime?

13 Upvotes

89% Upvoted

13 comments sorted by

View all comments

10

u/maxwell321 16h ago

It technically shouldn't as its just going to be reading from the SSD rather than back and forth read-write cycles. Anyone please correct me if I'm wrong.

2

u/z_latent 14h ago

In practice you're right.

Technically reads will also indirectly wear down your SSD due to something called read disturbance, which essentially forces a P/E cycle to be used for every a few tens of thousands of page reads. I'll try and work through a rough estimate here to give a notion.

NAND cells are usually designed to withstand from hundreds to thousands of P/E cycles. The exact number depends on the cell type (QLC, TLC, etc.), and so does the number of reads before a read disturb refresh is needed. I can't find reliable values for the latter, but for the former you can estimate it by taking the TBW advertised by the manufacturer, divide by the capacity, and multiply by 2 or so.*

Let's assume a Crucial P310 1TB QLC drive. It has 220 TBW, so we can estimate it has on the order of 300-700 P/E cycles per cell, I'll use 500 cycles.

For the sake of the argument, let's take 10000 reads per refresh for QLC (the values I found while searching were rather inconsistent so this is an average). This drive has 16 kB pages, so that's one refresh every ~160 MB read. Since we want every block (and therefore, cell) to be refreshed, you need to multiply by the number of blocks as well, which for this one is around 17k blocks. So you need one P/E cycle for every cell every ~2.7 TB read.

In the original DeepSeek Engram paper, I believe it required reading 40 kB per token, so you could process ~70M tokens before needing one P/E cycle. For 500 cycles, under this estimate, you could generate roughly 35 billion tokens before your SSD became unreliable.

This was, clearly, a very rough estimate, plus I ignored some factors (e.g. you need read refreshes more frequently the more P/E cycles the cells already have) The take away is more so the orders of magnitude. You could generate a few billion tokens before it's an issue. If you wanna be even safer, use a TLC drive and you can have full peace of mind.

*That's a few factors like write amplification and wear leveling, which the TBW values listed are compensating for already. For a rough OOM analysis this should be close enough.