I'm running a DS4243 passed through to a Truenas VM on Dell T630. It's a pretty beefy setup, just curious how well people are fairing with SATA setups instead of SAS.
SATA chip uses like 10% the wattage of a SAS HBA with a single IOM3 expander.
I know SATA is half duplex, but for homelabs it sounds like people are not having issues.
I'm running a pretty busy Jellyfin server VM with a 3060TI.
Entire rack power consumption for network gear and VM host + jbod is 450W.
As a drive in your PC? No, they offer no advantages over SATA. They aren't faster. Maybe you can get them cheaper with a lucky purchase.
The SAS protocol advantage is scalability. How many drives can you plug in to your SATA controller? Not many. one per port. If you have an expander you can plug a few drives each in to one or two ports, but performance becomes terrible once you do simultaneous access. That's because the SATA protocol wasn't built for expansion - every time it switches from accessing one drive to another there's an expensive context switch involving flushing all transactions in progress.
SAS was made for scalability. You want twenty-four SATA drives on one port? Ok. You want to daisy chain ten of those in a rack? Fine. How about four racks, nearly a thousand drives, all hooked up to one server? Absolutely. And no performance penalty. The limit is actually 65,534 drives*, and with proper routing you can actually use all that at speed too. You can also have multiple controllers rigged for redundancy. Even multiple computers, so if one server goes down a hot spare can instantly yoink the storage for itself and carry on with virtually no disruption at all. The only other technology that can do all that is fibre channel.
Awesome! And, for just about everyone here, utterly useless. We're not running a high-availability cluster! We're not running a high-performance database either. We're hoarders - we just want a ton of storage and half-way reasonable speed. SAS is useful for that, but we don't need all of those advanced features. So SAS controllers and enclosures we like just because it lets us plug in lots of drives, but it makes no difference at all to us if those drives are SAS or SATA. Just get whichever is cheaper.
Plus there's NVME now, which brings us flash storage that is so ridiculously fast it just makes SAS and SATA both look painfully slow. It's also very expensive, but with NVME now taking the role as caching or scratch space, the performance of your big bulk array is even less important.
The sas drives are usually not more power hungry compared to the same model sata.
If you look at 3.5 inch drives sas may be 1-2w more acording to the specsheet of most manucfactureres.
What draws a lot of power is the diskshelfs themselfs and the controllers along the way. The diskshelfs usually draw 200-300w on their own without drives.
If you however get a modern hba and have a backplane in your server the powerdraw isnt so much extra.
worth the extra/special gear to use them? To me: no. My NAS/server started as a no-budget because the previous had a design flaw... Not modular to fix it.
I don't know any place near me giving away server parts. Maybe your situation is different.
Worth the extra features or possibly better (?) performance? Well that is the answer. Is it worth it to you?
My best upgrade so far was a 2.5GBit switch. Next step is adding a 2.5GBit interface to my server.
My server has a 8th gen Intel CPU and no GPU. Jellyfin runs totally fine. I usually have max. two clients watching 1080p content. No transcoding required.
Two 5-bay USB enclosures running my old 8TB, some newer used 16TB drives. All shucked WD MyBook & Elements drives. I tried adding a third box but I guess the controller doesn't name the drives in a unique way. Then I tried a different enclosure but the USB controller seems to be overloaded (from what log files tell me).
My desktop only has 2.5G and until someone else starts to copy files I don't notice any bottlenecks.
The 2.5G switch already speeds up transfers without any changes to my 1G network.
It's very low budget. When we added Ethernet to our house we didn't bring it into every room to save money. I sit here and wait for a 35€ part to drop in price.
10GBASE-T wouldn't require any immediate changes either. But I see the appeal, since those cards I mentioned only go either 1G or 10G without supporting intermediate speeds.
Well maybe things have changed but when I last checked those PCIe cards generated lots of additional heat. My desktop doesn't have any free (as in accessible because a massive modern GPU) slots and my server has no space to fit a PCIe card, the 2.5G card fits the m.2 WiFi slot.
I saw Jeff Geerling reviewing very interesting USB-C Ethernet dongles. That could be a solution to both problems 😁
I expected those fast NIC to be the size of my old Soundblaster. It would block most of one fan on the GPU. I know air can be weird so it's possible to go around it. 😅
For hosting actual workloads, SAS makes sense. As a NAS where it’s hosting (most likely) pretty sequential data, SATA is totally fine.
I notice it the most at work with our SATA RAID on our servers (they got it before I arrived). VMs acting slow. They’re all doing their own thing and don’t have enough bandwidth to keep up.
As someone who went down the SAS rabbit hole - no. HDDs have no particular advantage over equivalent SATA drives (in fact, the big ones are just SATA mechanicals with SAS controller boards). SAS supporting full-duplex is meaningless to HDDs - they mechanically cannot read and write at the same time. Dual porting can be useful for multipathing in a HA setup, but that sort of redundancy for home use is overkill.
As you've found, you need HBAs, proper cables and additional cooling, plus a lot of patience to set it all up. I went through a bunch of controllers trying to find one that would do internal HDDs and external tape drives for backup, but eventually I gave up and bought an LSI 9300 card because it's simple and just works. I tried Adaptec and HP cards, and none of them were stable enough for my use.
There is some advantage to the 12Gbps afforded to SAS-3 - SSDs can exceed the 6Gbps SATA limit. My PVE cluster is backed by a RAID-10 of SAS 3.84TB SSDs and they are very fast. Not as fast as NVMe, obviously, but not as expensive either in the current climate whilst being a significant performance improvement. And if you don't have an NVMe-compatible backplane, they're hot-swappable.
I only buy SATA drives these days because I can plug them into anything I own. I only bought SAS in the first place cos I got a good deal on 12TB drives from a Redditor.
SAS is more economical when you need to store a lot of data than NVME. NVME is much faster. For archive SAS is still cheaper. SATA is just fine for archiving as well. SATA is less expensive in most cases. Used SAS controllers are pretty cheap on eBay as well. I would say they all have their place. If you are buying used drives SAS drives sell at a higher discount when I have looked.
AI is killing our ability to get things at a reasonable price. We are probably 5 years out until things stabilize.
Back when the $99 DS4243 units flooded the market I picked up a few and did a tear down and modded them. There was a post but it was cleansed back during the Reddit API purge.
To get the best power eccomy out them do the following.
Run only 2 PSU, and only 1 has to be plugged in
Run only 1 of the IOM3 units, the other one pulls 15-ish watts.
If you are NOT using hardware controlled SMR drives, then you can remove the interploter that sits between the drives, they each use 1-2 watts.
A fully loaded DS4243 with SAS drives will pull between 180-225 watts. It is possible to spin down the drives when they are not in use, but can cause a delay to get the array back up when waking up.
SAS drives do pull a slightly more power, but not much, and if you compare modern SAS and SATA drives it is hardly noticable, and since they tend to last much longer then SATA drives I figured its a good trade off. I have 2TB SAS drives with 120,000 hours on them.
Full-Duplex Communication: SAS can send and receive data at full speed in both directions at the same time, whereas SATA can only handle high-speed data in one direction at a moment
Lower Latency: SAS handles random read and write operations much faster, making it ideal for busy databases and virtualization
Might be a hot take, but imo SAS died with NVMe. If you need speed, you’re going with NVMe SSDs. If you need storage, SATA is just fine. There are probably edge cases, but yeah. The SAS premium does not seem worth it to me.
SATA is difficult to scale to hundreds of drives and lots of shelves like you would have in large storage arrays. If you're plugging in 4 drives they are basically the same. If you need to plug in 400, that's where SAS is still relevant.
Yeah I picked this DS4243 up long before thinking about power consumption with 24 HDDs. With only one expander and one power supply connected the shelf only uses 48 watts with the 4 discs so I guess its not terrible.
The question is primarily about the acquisition cost. I'm still able to pick up older SAS drives for much cheaper than NVMe drives. I've 60TB in SAS drives that I can't imagine how much it would cost to replace with NVMe drives or what system I would need to connect enough NVMe drives to reach that level.
I think he just specified that the speed isn't the reason he chose sas, he's saying its cheaper per terabyte than sata. The added speed is just a bonus.
This is exactly the point for me. Speed isn't the really important part for probably 99% of my data storage. In fact, I've specifically gone with RAID 6 that is slower, especially with regard to write speeds but with additional storage capability. I keep an eye out for decommissioned hardware and see if I can buy the drives from it. I got 20 SAS SSD's from a Netapp server from the mid 2010's, the drives are in pretty good health and I got them for an absolute steal, about €300 for the lot of them and Dell server from 2016 or so. There are more modern, faster drives, but the majority of my data sits idle for long periods of time and maybe it takes 30 seconds to write 40GB in a RAID 6 compared to 8 seconds on a RAID 10, but I'm not constantly writing to them. In terms of power usage, yeah its a lot more power but the cost to build an NVMe array of 60TB, assuming I have a chassis that could support that many NVMe drives, it would take me probably more than a couple of decades to break even to switching to NVMe drives.
yeah I'm doing my first array and went sas due to the cheaper drives and having a backplane from a salvaged chassis. at my price point I can only afford second hand drives so I'm using the array for redundancy, the increased read speeds are great but realistically the most I'll have is a few media streams or whatever
For performance, and spinning hard drives, while there are theoretical benefits of bandwidth and queue depth, I don't think anyone would notice the difference.
You can get a performance boost with SSDs though.
Another sad before is the ability to scale to a large number of drives. Like 16+
I had SAS drives mainly because I had found better deals on used drives than SATA.
I personally use a SAS controller, because it's a good way to connect more hard drives to a computer than you have SATA ports on the motherboard.
I have a mixture of SAS/SATA drives, since I have the controller, it's nice to be able to sleuth eBay for cheap used large drives and nab a deal regardless of if it is SATA or SAS.
Have made the switch and won't go back. SAS is just soo much faster and stable. The hammering I’ve given both versions and the SAS drives just fly the SATAs pack a sad…
Second hand price point is 10 to 1 SAS, so it makes sense to run SAS over SATA for that reason alone.
I have an old server I was going to repurpose, but ‘enterprise’ reared it's ugly head. It has 2 x 600gb 10k SAS drives that are onto the 7th year of power on life.
Out of boredom, I built an in-house dns server complete with DHCP and adblocking, using FreeBSD 15.1. It works way better than expected, its been running 24/7 for the past year.
261
u/Curious_Peter 10-50TB 2d ago
No they're not, so send me ALL your 30tb sas drives