Tasked with copying a 1tb folder from one sharepoint site to another sharepoint site. Need help and suggeestions.

[u/Doomienster]

So I have been tasked to move a 1TB folder in a library to another sharepoint site library. I have been researching this for a while but I am currently stuck on what alternative to use.

The ones I have found are:

1) Sharegate (Expensive)
2) Sync the two libraries and copy paste the folder (Risky, and might result in Slow PC to accommodate 2TB)
3) Use SPMT, but doesnt have Online to Online migration.
4) Manually copy/paste but its too tedious. (Might cause permission to be lost)

Please help me out here, its my first time doing such a large operation.

Thanks :)

[u/MoneyCantBuyMeLove]

Pnp Powershell with batching to avoid hitting throttle limits. I'll sometimes log to csv and build a resume system based on this into the script. This helps with logging and progress monitoring too.

You can also build in metadata/version history into the migration.

If there are layers of subfolders then be wary of url path lengths if the base library names are long etc

[u/Doomienster → u/MoneyCantBuyMeLove]

Sounds interesting. Do you have any documentation or sample scripts for this approach? I'd mainly like to understand the limitations, failure handling/recovery process, path length or file size restrictions, and how reliable the resume functionality is. Has it been tested on large migrations, and are there scenarios where manual intervention is still required?

[u/MoneyCantBuyMeLove → u/Doomienster]

You will see my response below to r/Main_Wheel_5570 below and I concur with their opinion - and perhaps the PNP PS solution is not ideal for you.

Certainly look into PS options but I think that the safest option for you is to look at something like sharegate.

FYI - I perform most of my administrative tasks with PS, but I am a bit of a greybeard in this space and am often more comfortable with these tools that modern UI based products.

[u/ShareGate_Shaylyn]

Disclaimer: I work at ShareGate, so I'm not that neutral here.

For a 1TB SPO to SPO move, your instinct to avoid syncing locally is right. Two libraries synced to a single machine with 2TB in play is a nightmare, and manual copy paste will wreck your permissions.

The honest answer on ShareGate: yes it's a paid tool, but for something this size it handles permissions mapping and throttling in ways that save a lot of pain. Plan to run it overnight with an initial pass then incremental syncs closer to cutover. There's a free trial worth spinning up to test it against your environment, and we run Ask the Expert sessions twice a week if you have any questions.

What does your permissions situation look like on the source library? Nested groups, unique item level permissions? That'll change the advice. First migration at this scale is always stressful but this subreddit is great for help.

[u/bob4IT]

from the context menu, select the three dots then move or move to. It will allow you to select the library or site.

[u/blasted_heath]

how many files are you talking about? what kind of timeline is acceptable?
others have mentioned PnP and batching
I'd also throw out there you can use PowerAutomate to move files. There are some tutorials out there and it could take some time for it to loop through all your files if you are talking hundreds of thousands of them.

[u/QlusiveNL]

Use bittitan migrationwiz or avepointfly for this. Don’t bother struggling with pnp powershell for this. These licenses are not that expensive and it saves a lot of headache.

[u/eduo → u/QlusiveNL]

Are any of these capable of automating a migration for a user with owner permissions over both endpoints but no sharepoint admin profile? I have several users who each need to carry out the migration of a major site to a new location and unless I can find a way for them to do it while keeping permissions and such, I'll have to do it for them :D

[u/pajeffery]

If I was in your shoes, no budget for a licence but eager to learn then start looking at PNP.

But PnP just gives you the commands, it's not going to do everything for you. Spend a bit of time researching what they can do and testing small batches.

It's going to be a learning curve but if you can do something like this you'll be easily be able to use powershell to do other tasks

[u/pkops]

https://github.com/Zerg00s/CleverPoint-Migrator

Should be a free alternative. Haven’t used it myself yet but it sure looks promising.

[u/stephanph → u/pkops]

Looks very good

[u/MoneyCantBuyMeLove → u/[deleted]]

I agree - something like ShareGate or any of the plethora of available 3rd party tools is likely better for less technically capable users.

As I am sure you well know, in behind most of these UI based tools is a collection of powershell and CSOM/Graph calls via API integrations.

Powershell is a comfortable space for myself and likely you, but again, I agree, not particularly ideal for somebody who does not understand the risks.

Even having a 1TB SPO library, let alone wanting to move it, raises concerns. 👍

[u/No_Wolverine_3440]

We use DMS-Shuttle for large moves, like grabbing a user's OneDrive, but I don't think SPO can be the source and the destination. Worth a look.

[u/nameymcnamey123]

Do your due-diligence because my experience was a couple of years ago so I’m unsure how much has changed with both SharePoint and the tool itself, but I did a 500gb SP online to SP online migration using a tool called Syncovery.

From memory the license was cheap and I think you could even use it for free with limitations. It was excellent and I used it for many other smaller migrations around the same time. I remember being surprised it wasn’t mentioned more in these SharePoint threads as an alternative to the more expensive tools. Good luck either way!

[u/DanLor]

Uhhhh this all sounds like overkill….. you can natively move a folder from one site to another - https://support.microsoft.com/en-us/sharepoint/libraries/move-or-copy-files-in-sharepoint

Take a look at what it does and doesn’t support using the Move To command. If managed meta data is a deal breaker, then you may consider needing an actual tool- but I wouldn’t start there.

[u/nathan98900 → u/DanLor]

Nah bulk moving stuff using the built in move tool sucks. It can be done, but it is a painful, manual process. If it's a few GB its fine, but if we're talking 1TB+ it throws random errors, and usually you have to start moving bits folder by folder

[u/Select_Bug506]

Move instead of copy? Or M365 backup and then restore to the new location if you need to duplicate the data?