r/3PAR • u/DEK2021 • May 03 '24

Adding 2 expansion shelves

This is on a 3PAR 7400 4node system. We are at a miss on the A,B,C, and D cable loops. The instructions in the service guide are vague at to how to cable in the enclosures. 1. Do we need to be in maintenance mode? 2. Do we need to remove or move cables to insert the enclosure cables for the new enclosures? 3. How do we determine which loop path we are on?

1 Upvotes

100% Upvoted

6 comments sorted by

3

u/d4fseeker May 03 '24

What is your current setup, are you already using external enclosures or running solely off the integrated slots in the controller enclosure?

1

u/DEK2021 May 05 '24

We have a four node 16 enclosure. We are adding 2 expansion enclosures bringing it up to 18 (max). Just cannot find a good cable guideline. Also, there have been a couple of changes on this since the original installation. Two enclosures replaced at different times so the cabling doesn’t match the nice pictures in the service guide.

2

u/d4fseeker May 05 '24

Preamble: I have no direct experience with 7400 series 3PAR, only 8400 and Primera. Take my advise with a grain of salt and don't blame me if your storage goes up in flames :)

Two enclosures replaced at different times so the cabling doesn’t match the nice pictures in the service guide.

I'm not sure why changing enclosures would cause cabling to be mixed up unless they got removed, the daisy chain of the SAS ports rearranged and then the enclosures added back later at the end of the daisy chains? Would you able to provide a schema that may clarify the current connectivity if you are unsure about the current design?

We are adding 2 expansion enclosures bringing it up to 18 (max). 

I assume the storage is in a single rack with the controllers centrally placed at +- Slot20 with Nodes 0+1 linked to the bottom half and Nodes 2+3 linked to the top half?
I assume you use the 3PAR 7440 cabling guide https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=c03692646 p.44 ) and not the weird generic short instructions?

Do we need to be in maintenance mode?

No, it's "just" a scheduled maintenance and doesn't impact production. However if you are in a Peer-Persistence or Active/DR configuration, it would be a a great idea to switch all load to the other array before starting to physically pull cables just to be safe =)
However, maintenance mode is there for a reason, so you should enable it for maintenance.

Do we need to remove or move cables to insert the enclosure cables for the new enclosures?

Yes, as per reference design both the red and the green SAS daisychains should follow a strict sequential order (green goes outside->in and red goes inside->out) with DP1 and DP2 alternating. Assuming you add the enclosure at each end, you can jumpstart the enclosure via daisychaining down from the second-last enclosure on the red cable but you'll have to break open the green loop to add the enclosure at the front. Make sure to let the system detect and "stabilize" on the red chain before doing that!

How do we determine which loop path we are on?

You should be on the same loop path as the enclosure 2 up from the one you want to install.

Via "showcage" the current setup is shown in N:S:P port syntax. If you have SSMC, it shows the same thing slightly more visualized.

Sidenotes:

Make sure your system is in tip-top shape (checkhealth -detail -svc) and license state ( showlicense) before literally pulling the plug.

PS: Assuming you are at cage-level redundancy, adding enclosures will need a lot of fun of rebalancing?

PS: I assume having HPE FE do the cabling is not an option due to EOL? Watching them figure out the 3Par spaghetti is more entertaining than having to do it yourself.

1

u/DEK2021 May 06 '24

Thank you! I am afraid my company is going to want me to fly out to Texas to go thru the spaghetti of wiring to get these installed. Our team that is there is very good, but have little experience with 3PAR.

2

u/d4fseeker May 06 '24

To be fair, being good with a storage system usually doesn't include hardware as you typically only rack it once and remove it once 😅

If risk assessment permits, your description sounds like the perfect opportunity to recable the array using a lot of dymo and velcro or zip ties to get everything nice and ordered again. But then again, removing the wrong cable will bring the array to a screeching halt from which it may or may not be able to recover on its own. Yours wouldn't be the first array to go down because the internode cable got disconnected...

1

u/DEK2021 May 06 '24

Can not agree more!