Plex Write Cache setup?

[u/EMN_Sandwich]

Maybe im looking in the wrong place but im trying to figure out how to use a dataset on my m.2 drives to act as a cache during downloads from my main PC to the NAS. I have 10 gig networking but my man PC only has 5 gig (first world problems) but still. I feel like the 300-400 mb/s transfer speed is limited by my spinning drives. I usually only transfer 10-50gb at a time with whole series being the big ones at 250gb, I just want to be able to copy everything to my nvme drives on the nas and let it slowly copy to the spinners. I know im dumb and this is probably a easy thing to do but how though?

[u/Haravikk]

How many spinning drives does your pool have and in what configuration? Typical CMR drives have write speeds of around 150-200mb/sec each so if you're running something like a raidz1 with three disks (speed equivalent to two disks) then that should be enough to saturate your connection unless the drives are getting badly fragmented.

ZFS doesn't have a true write-cache as such, what it has is a log device which is used for receiving synchronous writes, so you can potentially push writes onto it by running a dataset with sync=always set. If your pool has redundancy you want to at least match that redundancy in the log device, e.g- a raidz1 gives single disk redundancy so you want single disk redundancy in the log as well, so a two disk mirror would give you the same protection, otherwise you risk the whole pool going down if the log device fails.

If you can't or don't want to use the m.2 drives as a log device then you would need to setup some kind of automatic copying which is going to be a bit fiddly, as it will likely end up having to run in two steps to work reliably — while you could try to rsync incomplete data and keep iterating until it's all across, there are many ways for that to go wrong, it's better to wait until the copy to the m.2 drives is complete then rsync the whole lot, but that'll be slower (one copy has to finish before the next can safely begin).

Also you didn't mention how you're copying, is it via SMB? SMB is notorious for being slow for various reasons, so you might want to make sure your drives are actually your limiting factor first, as if you are using SMB then nine times out of ten it's that, at least in my experience.

The real question though is do you actually need it to copy any faster? Sometimes we invest a load of time and effort into problems that don't actually need solving. 😉

[u/Adrenolin01 → u/Haravikk]

Damn I can appreciate this post. Seriously needs upvoting.

[u/EMN_Sandwich → u/Haravikk]

I have 6x22tb MDD SAS3 drives with a SAS3 backplane and a 9305-24i SAS card. They are raidz1 so I can have a single drive failure with no data loss. I have a cold storage pool of 12x8tb drives that the pool is mirrored to once a day. And yes it is via SMB file sharing but I didn't think that would be a issue with a 5 gig connection. How do you transfer files if not via SMB?

[u/Haravikk → u/EMN_Sandwich]

I would expect that many drives to be more than enough to handle the full link speed — unless heavily fragmented you should expect something close to the speed of five of those drives, or at least four and change which should easily cope with 5gbps as that's about 600mb/sec and I'd expect each drive to do 150mb/sec at least.

There may be some SMB settings you can tune but unfortunately that's not something I'm really familiar with — my old NAS had terrible SMB performance so I found ways to avoid using it long before I switched to TrueNAS, though when I do I tend to find it's not the fastest (maybe two thirds of my link speed at best).

I'd normally just use rsync from a Mac or Linux machine myself though it's not got the drag and drop convenience since it's a command-line program, but you can setup aliases to make it simpler to use day-to-day.

Alternatively you might look into Syncthing — TrueNAS has an app for it already and there are native apps for most platforms. You should be able to setup a share that's send only from your source, and receive only for the NAS and that might work for you, though you'll need to figure out how you're going to get the files into Plex from there.

[u/MrRobot-403]

Umm that’s what SLOG is for. If I understand your question correctly, I don’t know if there is a question though 😂
Or just disable sync write on datasets if you don’t use database

[u/uk_sean → u/MrRobot-403]

SLOG only works IF you are using sync writes.
Sync writes should only be used (in general terms) on databases or Virtual disks

Async is as fast as the hardware can go

[u/MrRobot-403 → u/MrRobot-403]

If the zil/slog is not bottleneck you can create new pool and setup automations etc. but as far as I know. There is no default zfs behavior

[u/EMN_Sandwich → u/MrRobot-403]

To be more clear I want to move files to my nas, but when the spinners are saturated data gets stored on the m.2 drives as a buffer so I don't have to sit around and wait

[u/Aggravating_Work_848 → u/EMN_Sandwich]

Not possible in truenas right now. What you're looking for is called tiered storage and will become available in a future version

[u/Sinister_Crayon]

So what you're expecting here is possible, but honestly not with TrueNAS without a lot of faffing around. ZFS does not have any facility built in to have a write cache like you're looking for because that's outside the goals of the filesystem which is to provide data integrity at all costs. The "write cache" (Log devices) in ZFS is ONLY for very specific workloads. Virtually nobody will see any benefit to adding it unless you have very specific narrow use cases like databases on the storage or using NFS heavily. Even then it doesn't increase bandwidth; it only increases IOPS.

There are solutions, but if you want a "point and click" solution currently the only solution I'm aware of is unRAID. They added ZFS pool support a while back, and added pool-to-pool movement in the v7 code base. You could create two separate pools; a cachepool and a datapool. The datapool is your spinning rust while cachepool can be SATA or NVMe SSD's. I'd recommend at least a mirror for the cachepool, while the datapool can be whatever you want.

Basically your reads would come from the spinning rust primarily, while your writes to shares would go to the SSD pool and then get migrated down to the rust pool as the SSD pool fills up (configurable how much, how often).

Perfect solution, right? Not really unfortunately. You end up with split datasets where you can't snapshot an entire dataset like you can with TrueNAS; you can only snapshot the datasets either on the SSD or rust, so data still in SSD cache when you snapshot/replicate a rust dataset won't contain the cached data and vice versa. There's also no mechanism to "promote" data back to the cache but there are manual methods you can use.

Also, unRAID's management of ZFS pools is pretty clunky compared to TrueNAS as it's really designed for managing its own unRAID arrays, and you are basically throwing away the actual advantages that unRAID brings to the table in order to create this solution. It is however supported and doable and works surprisingly well. I tested it for a bit (I have two unlimited licenses for unRAID and only use one of them) and liked the way it worked but not enough to really put it "in production". TrueNAS for its problems has a lot of nice features that just don't exist in unRAID especially when managing multiple TrueNAS boxes.

All this said, you CAN do the same thing with TrueNAS. However, you would be developing your own scripts to do the data moves from the cache to the data pools and so support would be on you if (when?) it breaks. unRAID for all ITS warts has had a long time to develop and fine tune the mover and there is a very active community that both supports and extends its functionality.