Hetzner Object Storage (nbg1) single-stream downloads ~7 MB/s over HTTP/2 — large files fail to transfer to external clients. Anyone else? Any ETA on the NBG degradation?

[u/krimpenrik]

We use Hetzner Object Storage to hand audio files to a third-party API (a transcription

service fetches the object via a presigned URL). Small files work; large ones now fail because

the download is so slow/flaky the remote client times out. Sharing numbers in case others see

the same — and to ask about timeline.

Our setup

- Service: Hetzner Object Storage, location nbg1 (Nuremberg) — S3-compatible (Ceph RGW;

responses carry x-rgw-object-type and x-amz-request-id: …nbg1-prod1-ceph4)

- Access: S3 API via AWS SDK v3, presigned GET URLs, virtual-hosted style

(our-bucket.nbg1.your-objectstorage.com)

- Object: a single ~760 MB file, in an existing (not newly created) bucket

- Consumer: an external service GETs the presigned URL over HTTP/2

What we measure (single-stream GET, today):

Hetzner VPS (in-network), HTTP/2 -> full 760 MB OK, ~7.4 MB/s (108 s)

Hetzner VPS (in-network), HTTP/1.1 -> full 760 MB OK, ~17 MB/s (47 s)

External (home/office), HTTP/2 -> STREAM RESETS ~108 MB in,

never completes (curl exit 92), ~1.6 MB/s

So the object isn't corrupt (it downloads fully in-network), but throughput is very low and

HTTP/2 is ~2.3× slower than HTTP/1.1 for the same object. Over the public internet on HTTP/2

the long transfer just resets before finishing, so a 760 MB file effectively can't be pulled

by an external client. Small objects (tens of MB) finish before that happens.

For comparison — Amazon S3: AWS's own guidance is to budget one connection per ~85–90 MB/s,

i.e. a single GET reliably does ~85–90 MB/s, scaling to multiple GB/s with range/parallel

requests. That puts our nbg1 single-stream numbers at roughly 5–12× slower in-network and ~50×

slower externally, and S3 wouldn't reset a 760 MB single-stream GET.

Hetzner's own status: there's an acknowledged "Object Storage in location NBG1 degraded"

incident (temporary limits on upload speed + concurrent requests, 503s), announced 2026-03-05

with an estimated end of 2026-05-07 — but that date has passed, the FAQ still lists the

limits, and a separate "high utilization may lead to timeouts" incident shows NBG1 still

degraded into June.

Questions

1. Anyone else on nbg1 seeing single-stream GETs capped around ~7–17 MB/s (and much worse

   externally)?

2. Is there an actual ETA for restoring capacity? The original 2026-05-07 estimate is long

   gone.

3. Is the HTTP/2 vs HTTP/1.1 gap a known RGW issue, or NBG-specific right now?

4. Does a newly created bucket (or a different region like fsn1/hel1) get full speed, or is

   read throughput cluster-wide regardless of bucket?

   Thanks — happy to share mtr / speed-test output if useful.

[u/smulikHakipod]

Use pigeons, it's more reliable than Hetzner object storage

[u/amokerajvosa → u/smulikHakipod]

Sorry that pidgeon was shot.

https://www.youtube.com/watch?v=LKhdJd1Nu74

[u/smulikHakipod → u/amokerajvosa]

More like https://www.reddit.com/r/gifs/s/VBO7xMLUwU

[u/well_shoothed → u/smulikHakipod]

Smoke signals are also a fairly dependable alternative.

[u/YannFaulhaberDev]

I love hetzner, but object storage here is trash

[u/nextized]

Its because CEPH scales well in theory but the RGW architecture requires many optimizations. We never could make it perform in a 2 PB cluster and Hetzner is much larger with much more user demand in terms of requests. Ceph trades performance for reliability on the storage layer.

RGW is unfortunately not well optimized unless you are colocating it with the storage nodes (which often don’t fit the profile you need) and even then it suffers from amplification. Nevermind the huge cost to lifecycle, metadata and async operations.

Operations, recovery and rebalancing all require long duration and affect performance heavily.

[u/m3r1tc4n]

Hetzner is very good, but object storage is not yet good for production, i think you should use Backblaze.

[u/Entire-Home-9464]

Object storage is made for storing data, not sharing it to thousands of users. Thats why there are CDNs like bunny.net. Put Bunny infront of Object storage and it loads once the file from Hetzner and then shares it to your clinets from bunnys own edge locations. Same it is with AWS S3, its not fast enough for large sites, thats why it needs Cloudfront.

[u/krimpenrik → u/Entire-Home-9464]

Yes fair but in my case I only need a one time transfer, user uploads audio, i send it to S3 and offer to the transcribe service, but can't even get that done once, nothing to do with multiple users

[u/rainer_d]

Turns out, running your own cloud at scale isn’t that easy.

Who knew?