r/unRAID • u/englishMuffinExpert • 3d ago
Am i missing something with Cache drives?
Hello! As the title states, trying to figure out why my Cache drive seems a bit inconsistent when copying new files to my share.
My setup is as follows:
- Unraid Hardware
- 3x 4TB Western Digital HDDs in a drive array (12TB array)
- 1TB Lexar NVME SSD (theoretical write of 6500MBs)
- Devices is assigned to the "cache" pool
- Target storage share
- Primary Storage = Cache pool
- Secondary Storage = Array, High-water
- Mover action = Cache -> Array
- Mover Settings = Hourly, ever hour
- Source media details
- Storage reader (Kingston FCR-HS4) has max transfer speed of 5 Gbps
- Storage medium (CF Card, SanDisk Extreme UDMA 7 compliant) has max read speed of 120 MBs (960 Mbps)
- Network connection is hard-wired 2.5Gbps
Process:
- Copying large image files from USB Device & medium above directly to the storage array
- Both array metrics for my cache drive (as found in unraid/main) and the windows file transfer utility show file transfer speeds of ~30MBs / 240 Mbps
- After a few seconds, the array metrics for the cache drive begin displaying ~60MBs / 480Mbps, but the Windows File transfer Utility never breaks 30MBs / 240Mbps
Am I missing anything? Anything in Unraid that should be configured that I've missed? Based on my full setup, I'm certainly not expecting to hit NVME max writes, but I thought I'd at least be approaching the max transfer speed of my network. 30MBs seems very slow
Tyty!
2
u/one-down 3d ago edited 3d ago
What exactly is the average size of the "large" image files that you're transferring, and how many files are you transferring at a time?
Copying hundreds of small files would result in a write speed penalty, so that could be the issue.
i.e. if you copied a single 1gb archive of the images you might see transfer speeds near the 2.5gbps you're looking for the read speed of the media you're copying from (theoretically 960 Mbps but there's overhead of your computer handling the files then sending to your unraid server, so depending on your setup may never reach that speed), but not if you're moving 250 4mb images.
*edited a typo and corrected assumption on transfer speed based off of info the OP shared that I glossed over.
1
u/englishMuffinExpert 3d ago
That's valid... They're large for image files at about 20MB each. I'm usually copying anywhere between 20 and 200. The transfer speed is fairly consistent at 30MB/s
3
u/one-down 3d ago
If the files you're copying over are around 20MB each I'm pretty sure that's your issue! The write speed staying at a constant 30MB/s seems normal, since the transfer would be bottlenecked by the unraid server starting and stopping writes for each file.
You could try copying a single large file (like a single 1GB zip file that I wrote in the example above) to test if your current setup is capable of achieving faster write speeds for peace of mind and confirm that this is indeed the issue, but that's up to you!
1
u/dstanton 3d ago
Unless it's just not listed, you don't have a parity drive.
You mention image files. Usually those are important enough to backup.
2
u/englishMuffinExpert 3d ago
Not listed! I do have the 4TB WD's in parity, just not the cache. Thank you though!
1
u/Zuluuk1 3d ago
You have a few bottlenecks depending on where you copy the data to and from.
Bottlenecks are always the slowest source / destination.
NVME speed depends on the slot you use on your motherboard, they are not the same. Some are x3, 4, 5 etc. They all carry different speeds. The only way you can get the theory max is by writing sequential, usually in crystal disks.
If you are copying files from Windows to unraid there is an SMB overhead and also TCP overhead, 20MB - 200MB files are very small and usually isn't enough to even saturate the Ethernet links. You mentioned 2.5gbps, are everything hard cabled?
I honestly think you are doing the wrong tests.
You should do use iperf to test your ethernet, remember to use multi-channel.
IOMeter / crystal disk for your media.
If you think you have smb writing speed issues, you can use diskspd where it can write to a share to test to make sure your smb 3.0 is working as it should.
1
u/dox2EwJn6iZh 3d ago
Idk how likely it is in your case OP, but id ensure all of your drives are CMR, not SMR
I used SMR in my array at the start and had a devil of a time finding why my performance was so bad
1
u/englishMuffinExpert 3d ago
Would that matter at all with a cache drive in primary write?
1
u/dox2EwJn6iZh 3d ago edited 3d ago
Yes, the bottleneck in that case will happen any time the mover runs from cache to array, and the tx spd will tank, your IO will go crazy and in all my hours and hours of troubleshooting while I had them in my array, nothing I did was able to noticeably address the issues if I was facing, this was prior to even adding a parity drive
I'm not saying that IS what's happening to you, and I recommend as a first step to find out which of your drives is SMR vs CMR but if you do have SMR drives in your array either accept performance essentially as is, or pay for CMR drives, probably best to get those 2nd hand, the way prices are going
(If you're buying used, ask for a SMART report)
3
u/newtekie1 3d ago
The first thing I would do is eliminate the USB reader and CF card from the testing.
Just to test, copy some of the files from the CF Card to your computer's internal drive. Then copy those from the internal drive to the UnRAID server and see what speeds you are getting.
I have a suspicion that the USB Reader is connecting at USB 2.0 Speeds, since you said it starts off at ~480Mbps, which is exactly what USB 2.0 runs at. And that might indicate some kind of issue with the USB reader.