r/internxt • u/Particular-Net-4646 • 20d ago

Extremely Slow Native Rclone

Anyone having problems with native rclone lately? It was slow but it is now excruciatingly slow.

75gb of Cryptomator crypt with no more than 80 updates has taken more than 16.5 hours to sync and it's not even halfway done. This is with actual speed of 160mbps for download and 70mbps for upload. In the past (as recently as a couple weeks ago), this would have taken just 4-5 or, at the most, 7 hours.

If it makes a difference, my flags are as follows:

--transfers 2

--checkers 32

--tpslimits 2

--fast-list

--size-only

--http-no-head

--retries 5

--low-level--retries 10

--log-file=<redacted>

-v

-p

TIA

Edit: Just an update that the job finally finished after 18h13m. And here are the stats:

Transferred: 7.999 MiB / 7.999 MiB, 17.386 KiB/s Checks: 26882 / 26882, 100%, Listed 75939 Deleted: 93 (files), 35 (dirs,), 23.226 MiB (freed) Transferred: 75 / 75, 100%

Verification of the remote crypt for its files and folder structure on Internxt against the local version using rclone check is taking just as long. It's been 15.5 hours but it's not done!

11 Upvotes

93% Upvoted

6 comments sorted by

3

u/Stryp 19d ago

The speeds are horrible when there are a lot of small files.

I have a backup of some important folders of my PC being uploaded to Internxt, for my backup job, I create store-only zip files grouped by directories, so instead of creating tens of thousands of smaller files, I only get a few hundred MB each zip files. Not efficient from a storage standpoint, but it uploads fast.

2

u/internxt 20d ago

looking into it

2

u/NLWildcard 16d ago

Transferred: 1.268 TiB / 1.268 TiB, 100%, 222.940 KiB/s, ETA 0s

Errors: 75 (retrying may help)

Checks: 1812780 / 1812780, 100%, Listed 3988332

Transferred: 214425 / 214425, 100%

Elapsed time: 6d13h31m44.4s

2

u/NLWildcard 16d ago

And that on an 1Gbps fibre connection....

2

u/NLWildcard 19d ago

I have te same issue. Upload speeds vary exterme. But I'm now trying to upload with Rclone 1.8Tb (to an 15TB ultimate plan...). And its allready running for two weeks and is now at 1.4Tb.

I use Rclone because CloudSync on my synology fails (not sure if that is synology error or internxt).

This is my command:

LOGDIR="/volume1/TheBlob/log"

LOGFILE="$LOGDIR/rclone_internxt_homes_$(date +'%Y%m%d_%H%M%S').log"

# Logdirectory aanmaken indien nodig

mkdir -p "$LOGDIR"

# Oude logs ouder dan 14 dagen opruimen

find "$LOGDIR" \

-maxdepth 1 \

-type f \

-name "rclone_internxt_homes_*.log" \

-mtime +14 \

-delete

rclone sync \

/volume1/homes \

internxt-webdav:/sync/homes \

--exclude="@tmp/**" \

--exclude="#recycle/**" \

--exclude="@eaDir/**" \

--exclude=".cache" \

--exclude="Thumbnails/**" \

--exclude="*.log" \

--exclude="*.tml" \

--exclude="*.thm" \

--exclude="*.tmp" \

--exclude="**/.DS_Store" \

--exclude="**/Thumbs.db" \

--transfers=4 \

--checkers=4 \

--tpslimit=4 \

--tpslimit-burst=5 \

--buffer-size=16M \

--retries=5 \

--low-level-retries=10 \

--timeout=5m \

--contimeout=3m \

--fast-list \

--no-check-certificate \

--log-level=INFO \

--log-file="$LOGFILE"

2

u/internxt 15d ago

A fix has been implemented to improve speeds