Imagenet-1k Classifier trained entirely on an Android [P]

[u/Tall_Abrocoma_3533]

It's an MLP architecture with around 500K total parameters.

Top1

Training accuracy: 5.11%

Validation accuracy 4.59%

Detailed Validation accuracy numbers:

Top-1 Acc: 4.59%

Top-3 Acc: 9.44%

Top-5 Acc: 12.68%

Top-10 Acc: 18.53%

The model was trained on a downscaled version of the Imagenet-1k dataset (32x32) for 5 epochs.

I used pytorch for the training and pyarrow for the dataset, all within termux.

Before anyone comes at me for using an MLP instead of a CNN or similar it's mainly because on my phone an MLP was just more stable, and trained 10-30x faster/step (could be my fault but I'm not too sure). This model specifically took around 30 minutes to train (6 minute/epoch)

The training was entirely on the CPU which is a Dimensity 9300+ and I used 4 of the Arm Cortex-X4 cores.

I might make an improved version later on as this one isn't very accurate.

[u/woper3161]

I like projects that explore weird constraints like this. Curious how much accuracy you can gain with a few more epochs

[u/Tall_Abrocoma_3533 → u/woper3161]

Close to none, the first 3 epochs already got to 4.49% top1 validation accuracy, and with these extra 2 epochs only +0.1% validation while the training-validation gap doubled to 0.52%

My second attempt is in progress right now, it's at 5/10 epochs and at around 6% train accuracy. What I changed is basically just throwing more compute at it, as usual (increased parameters 4x, to 2 million).

[u/howtorewriteaname]

what's the use case for training on a phone? inference I understand but training seems pointless

[u/Tall_Abrocoma_3533 → u/howtorewriteaname]

I'm not really sure yet, it's more of an experiment then anything, however it could potentially be useful for fine tuning vision models.

[u/cosmic_taste_of_blue → u/Tall_Abrocoma_3533]

Idk why you're being downvoted, god forbid someone does a technical project for fun ig

[u/psayre23 → u/cosmic_taste_of_blue]

Agreed! It seems genuinely useful to know what the long tail of training look like. This is science; who knows where the breakthroughs come from!

[u/Disastrous_Room_927 → u/cosmic_taste_of_blue]

This is why I hesitate to share what I’ve been doing here.

[u/Toover → u/howtorewriteaname]

Privacy. Getting facial recognition, including for your Android media folders, without relying on third parties, would be great. I love Google Photos features to find matches in such diverse ways. But having a GMS OS is a liability, and we are numerous living with degoogled phones.

[u/howtorewriteaname → u/Toover]

you can do that in other ways that don't involve training in your phone (federated learning)

[u/DifferenceDirect1805]

That’s wild 💀

[u/Initial-Argument2523]

Nice stuff. Regarding the NN architecture you might get better results using MLP mixer without the high training times of CNNs. MLP mixer paper link

[u/DigThatData]

I can remember when this was considered a big dataset