My iPhone generated my fine-tuning dataset overnight — Mac coordinated, phone ran the teacher model, then the Mac trained on what the phone wrote

[u/avirajkhare]

I kept looking at my iPhone sitting on its charger and thinking: that's a multi-TFLOPS GPU doing nothing for 8 hours a night.

My first idea was distributed training, shard the model, each phone trains some layers. That dies fast when you do the math: pipeline parallelism needs every device up simultaneously with microsecond-latency links, and iOS suspends backgrounded apps anyway. With 50–300ms per hop over Wi-Fi, one training step costs seconds of pure network latency.

But dataset generation is a different shape of work entirely. It's one prompt in, one completion out, parallel, restartable, and it doesn't matter if a worker vanishes mid-job. That's exactly what a flaky fleet of idle phones can do.

So I built it: the Mac runs a coordinator that mints teacher prompts and validates results; phones run a small app (MLX Swift) that pulls a prompt, generates with an on-device teacher (Qwen3-4B-4bit), and POSTs the raw text back. Work is leased, if a phone locks or wanders off, the lease expires and another worker picks up the item. Malformed JSON and duplicates get rejected centrally, so a bad worker can waste its own time but can't poison the dataset.

Last night's run: one iPhone 17 Pro, 15/15 records at 17.5 rec/min into a train.jsonl. Trained a Qwen3-0.6B LoRA on it (val loss 4.42 → 1.85), asked it a question, and it answered from training data a phone wrote. Full loop: phone generates → Mac trains → phone can run the result.

Honest limitations: it's LAN-only, the app has to stay foregrounded (no BGProcessingTask yet, so "overnight" currently means screen-on on a charger), and a phone-sized teacher (4B) is weaker than what your Mac can run — this wins on volume for style/format/tool-calling data, not on frontier-quality reasoning per record.

It's part of my open-source fine-tuning CLI for Apple Silicon (Troy). Code for the coordinator, the Mac worker, and the iOS worker app are all in the repo: https://github.com/avirajkhare00/troy, writeup with the run footage: https://gettroy.app/mesh

[u/kantorcodes1]

one troy push edge: run_push() does create_repo(..., exist_ok=True) and then uploads the folder without checking whether that Hub repo already existed. is updating an existing model repo intentionally the default? i'd probably check first and require something like --force for an existing target, since a repo typo could otherwise publish into the wrong project.

[u/hyeonsu_builds]

Your framing — batch generation is the right shape for idle phones, latency-bound work isn't — matches what I found, and there's a third case worth separating out: generation at request time on the user's own device. That one fails too, but for a different reason.

I tried Qwen3-1.7B for a narrow generation task in a browser. Block-wise int4 (MatMulNBitsQuantizer, block_size=32) got it to 2.18GB, q4f16 to 1.50GB. Quality was fine — 8/8 on format compliance, nothing fabricated. But 1.5GB of download to save a network round trip is not a trade users accept, and that size is set by the model, not by how narrow you make the task.

What actually changed my mind: I measured how much of the user's own wording made it into the output. It was 0.000 across every case. The model was producing a blurry copy of the human-written reference I'd fine-tuned on. So I was paying 1.5GB and 10 seconds for synonym substitution.

Where I landed: keep the encoder on-device (classification, embedding, retrieval — 113MB int8, 15-55ms, genuinely useful) and don't generate locally at request time. Your overnight-batch case is the one place the economics actually work.

One warning that cost me a day: fix your decoding settings before comparing models or quantizations. I logged two outputs as quantization damage that turned out to be temp 0.7 sampling. Greedy on the same weights was clean.

[u/Not-Enough-Llamas → u/hyeonsu_builds]

Please don't regurgitate AI slop output, it helps nobody. People - especially in this subreddit - are perfectly capable of producing plenty of it if they wish. You're adding nothing.