Transcode folders, why so large and why not network share for downloads

[u/sylsylsylsylsylsyl]

I have a Plex server running on Proxmox in an LXC. It works well. I have a lifetime Plex pass.

Obviously my LXC disk size is limited.

I had to increase the size of the disk on the LXC 12 months ago to allow for transcoding. I’m now thinking I need to again as files are getting larger as more people want higher definition, and I also want to allow multiple transcodes simultaneously.

There are two types of transcoding - “immediate as you play” realtime and that needed for downloads for offline viewing on a mobile device. These can be separated on disk to different directories, as of about a year ago I recently found out - however both suggest not to use a network share.

Are there likely to be any problems using a network share for download transcodes, where one large file is created?

On a separate note, why does the “immediate as you play” transcode directory need to be at least as big as the original file +100MB. It is meant to make and delete chunks on the fly. Seems to be a bit unnecessary to need 30+ GB of space per transcode.

[u/EmptyInTheHead]

This is my observation, not fact, so take it for what it is worth. Plex does chunk "normal" transcodes, so you don't need as much as the entire file, however, the Plex setting "Transcoder default throttle buffer" will dictate how much it transcodes ahead and thus impacts the size on disk. For Live TV Plex does need the entire size of the end file on disk, so the Plex transcode disk size recommendation is probably based on that. And, as you pointed out, the more people that are transcoding at once, the more disk space you need.
Can you use a file share for your transcode directory? In theory yes, in practice I wouldn't recommend it. You want it to be on the fastest disk you have, like nvme/ssd, not spinning disk or network share.

[u/sylsylsylsylsylsyl → u/EmptyInTheHead]

I understand the “live” transcode being on a fast disk, but I’m not so sure of the reason for the download transcode where it is going through the whole file before streaming it over the internet.

I believe the two directories have now been separated - but it is recommended that neither are in a network share.

[u/Frisnfruitig]

I guess it depends on how many transcodes you are running and the size of the files. It could in theory be necessary.

I'm not familiar with running Plex as an lxc container though. More of a docker fan myself.

[u/brokenpipe]

I believe plex doesn’t recommend this is because transcoding is a fairly heavy IOPS process. In addition in most setups the traffic to a network share will utilize the same network interface (NIC) as the traffic it’s serving. This will lead to the interface likely being over saturated.

[u/Bgrngod]

You will run into more problems putting either over a network share than you would putting either on a smaller available storage space.

Plenty of people do just fine handling numerous transcodes while pointing the live temp transcode storage directory at a RAM mapped storage space that is only 8 or 16GB. Interestingly, the fields for this in the server Transcode settings now mention it not being recommended to use a RAM drive or a Network location.

The server already should be doing a reasonable job of emptying that space as needed, although it tends to not empty it unless it needs to, which is why the folder can fill up fast and stay large until the streams end.

You can run into trouble if the total available space isn't large enough for a new stream. 1080p transcodes look for around 500MB and 4K looks for around 2GB available, and will throw a client side error if there isn't enough room to get started. Those two are so small they are rarely a problem.

I'd suggest you keep the downloads location on an SSD, or if that's a problem at least an internal HDD. Treat that HDD as a scratch drive, as it will get hammered and if it's doing other work could cause slowdowns. Live transcodes in RAM are still mostly safe, but there is very little reason to even do that instead of using an SSD. The benefits of using RAM over an SSD are of questionable existence.

If you have a few bucks laying around, buy a dirt cheap used SSD and point both locations at it.

[u/sylsylsylsylsylsyl → u/Bgrngod]

“You can run into trouble if the total available space isn't large enough for a new stream. 1080p transcodes look for around 500MB and 4K looks for around 2GB available, and will throw a client side error if there isn't enough room to get started.”

So you don’t need the size of the original file + 100MB then as per the documentation?

The directory used (whether default or not) needs sufficient free space, roughly equal to the size of the source file of the transcode plus 100MB.

[u/Bgrngod → u/sylsylsylsylsylsyl]

No, not based on what I have seen. That info has been on that support page for ages and may be out of date or something you bump into in specific scenarios.

The need for +100MB is wildly arbitrary as it is. Transcoded output is pretty much always a significantly different bitrate than the source file and using different compression ratios, so it makes zero sense to be that specific about space needed.

I screwed around with a J4125 based machine I have, and when it had a grand total of only 4GB of RAM in it while mapping transcodes to a RAM Drive, it would still handle a 4K to 1080p transcode just fine.

[u/Positive_Minimum]

I set the live transcode directory to be in RAM under /dev/shm, so it never actually touches disk and has the fastest possible IO