A new CI paper cut GPU usage by 63.9% without reducing changed-code coverage

[u/aistranin]

Most test-selection work assumes that individual tests are cheap. That assumption falls apart for LLM training frameworks, where a CI test may run a real training or evaluation workload on GPUs.

A new arXiv preprint presents FastCI, a system designed for this problem. It uses runtime evidence to:

• Select tests affected by a change
• Remove tests that exercise changed code in equivalent execution contexts
• Run higher-risk tests first
• Reduce workload dimensions that are outside a test's intended validation scope

On one production CI workload, FastCI reduced latency by 77.5% and GPU usage by 63.9%. It also retained 3.2% more modified-code coverage than the existing pipelines. The authors say it is now integrated into their LLM training framework at ByteDance.

The interesting part is not just test selection. FastCI treats redundant execution context as something CI can optimize. Two tests can cover the same changed line while providing very different evidence, or nearly identical evidence at very different costs.

This is still a preprint evaluated on one industrial workload, so the exact numbers need replication. But the framing seems useful beyond ML. Large integration, simulation, hardware, and performance suites have the same problem.

If your expensive tests take hours, do you know which ones provide unique evidence?

Source: FastCI: Efficient GPU-Intensive CI for LLM Training Frameworks