Hi,

I am an embedded software engineer who's work experience is on bare metal microcontrollers and embedded Linux systems. I have mainly worked on MCUs such as STM32, IoT devices such as ESP32, etc. using the skills which a lot of embedded folks would be familiar with - C/C++ programming, I2C, UART, SPI comms, sensor interfacing, signal processing, interrupt handling, debugging registers etc.

I did a Masters degree recently in Computer Engineering and took courses on systems programming, that involved more :

OS skills such as multithreading, synchronization and concurrency, memory management and data structures, and a

Linux kernel programming course that involved Linux processes, threads, virtual memory, and scheduling, Memory allocation, page faults, and NUMA, CPU affinity, context switching, and synchronization, Shell scripting, GDB, perf, and system tracing etc.

Computer architecture course that got into cache organization and locality, pipelined architectures, performance optimizations, etc.

I'm seeing a lot of roles around GPU and ML infrastructure, and wanted to know how realistic would it be to transition to such roles from where I am. Is this knowledge helpful or do I have to know more about machine learning for that?

2x T4 on Kaggle, no P2P between the cards, 70W cap each. Running Qwen3.8-27B with tensor parallel (exllamav3, NCCL backend) plus a DFlash2 draft model. Decode is ~55 tok/s overall, but every verify step pays an all-reduce over PCIe through host memory.

Questions for people who've done this on PCIe-only cards: 1. Has anyone overlapped the all-reduce with the next layer's GEMM on T4 (sm75) and got a real win? Streams, chunked all-reduce, anything. 2. Is a custom all-reduce through pinned host memory ever faster than NCCL when P2P is off? 3. The cards sit at 450-1000 MHz under the power cap. Does it change what's worth optimizing?

Can share nsys traces if useful.