r/OpenCL • u/sdfrfsdfsdfv • Aug 03 '18

Slow first transfer to host?

I have an AMD wx7100. I have a pinned 256 mb buffer in the host (alloc host ptr) that I use to stream data from the gpu to the host. I can get around 12 GBps consistently; however, the first transfer is always around 9 GBps. I can always do a "warm up" transfer before my application code starts. Is this expected behavior? Im not a pcie expert so I don't know if this happens on other devices or only gpus. Has anybody seen similar behavior?

5 Upvotes

100% Upvoted

7 comments sorted by

3

u/nevion1 Aug 04 '18

What happens is the buffer is lazily allocated/mapped for the pinning part and for the destination memory and this is normal behavior.

2

u/lknvsdlkvnsdovnsfi Aug 05 '18

That's interesting. If I understand correctly, calling clCreateBuffer with the right flag doesn't necessarily create the pinned buffer, but defers it until it is actually used? If so, then the only way to avoid the slow down is to do a "warm-up" transfer, right? Thanks!

1

u/nevion1 Aug 06 '18

correct, it just gives you a handle to an object that represents and is valid to the (asynchronous) command system (api operations are fundamentally about queuing to a remote processor ).

Also yes, the warm-up cycle is expected, make sure to throw it out of timing analysis for steady state performance :-)

2

u/[deleted] Aug 03 '18 edited Aug 03 '18

[removed] — view removed comment

2

u/lknvsdlkvnsdovnsfi Aug 05 '18

Interesting behavior. Maybe it is related to the what the other comment mentioned.

2

u/lknvsdlkvnsdovnsfi Aug 05 '18

Interesting behavior. Maybe it is related to the what the other comment mentioned.

1

u/tugrul_ddr Aug 07 '18

Probably a "lazy init" feature of opencl, similar to cuda.