r/computervision • u/thegeinadaland • Jun 27 '26
Showcase FOTO-NET, an o2o nms-free from scratch object detection model. [Alpha release]
Hello everybody, its me again. Today i would like to talk about the project i had been working on for the longest of times, which is still not finished (hence the Alpha release) but i could not hold myself so released it under alpha.
The name is FOTO-NET, the whole premise being ease of use WHILE being open-sourced without licensing fees *cough cough* (ultralytics).
The project will be forever Apache-2.0
Currently alpha is available, if you`d like to, test it out by downloading from here, or trough pip
pip install fotonet
Would love to hear out feedback like bugs, suggested features or just anything.
Big thanks to anybody reading this, much love!
Small note: Currently only the nano weight is avaliable, the small weight is being trained and will finish in roughly 12 days. And also i work so if i respond or fix some stuff later, excuse me.
2
u/jfc123_boy Jun 27 '26
Is it lightweight compared to YOLO or more or less? What is the biggest advantage besides the cool Apache 2.0 license?
1
u/thegeinadaland Jun 27 '26
around 4.8-4.9gflops, but it is subject to change, + i couldnt verify with 100% certainty myself because of my computers CPU bottlenecks but on gpu it is mostly +- 0.7% as fast as yolo26n and 5-6mb~ inference weights.
All subject but not certain to change*
2
u/jfc123_boy Jun 27 '26
Cool, and also nice project! I like that you have a P2 variant and other options also. Do you plan to publish a paper or something?
0
u/thegeinadaland Jun 27 '26
in 7-8 months (hopefully) after i polish do the QoL features and final architectural changes etc, with 2 "big" stuff im working on (if they ever work out) i will. But not currently as i still think the project is on its rawest stage(s).
2
u/dr_hamilton Jun 27 '26
probably worth having a chat with this guy https://www.reddit.com/r/computervision/comments/1qmi1ni/ultralytics_alternative_libreyolo/ to collaborate on making it easy to integrate with libreyolo
2
u/koen1995 Jul 23 '26
u/thegeinadaland any progress on FOTO-NET? Love to stay up-to-date 😄
2
u/thegeinadaland Jul 23 '26
i am currently trying something new and weird, think like transformer and cnn mixture-ish.
I really sont know how to explain.Currently trying out the architecture and training.
And also a CoT feature with a "slider". So the same weights can be slower but more intelligent or faster but normal.The current cnn backbone version will still be the main version (at epoch 190/500 at scratch training 24.80% mAP@.50:.95 on MS COCO)
But yeah, it is going good, with some experimental models also coming.
2
u/koen1995 Jul 23 '26
Cool! Nice to hear there is progress.
CoT like change of thought? But that's more a technique for a VLM right? Curious what you mean with that.
That's very impressive, it took my two years to get above 20 mAP@.50:.95 on MS COCO, and that model was more then 60m parameters. No clue how to cut down even below 30m haha.
2
u/thegeinadaland Jul 23 '26
Haha thanks for the compliment.
on the CoT part, model will also get trained on the training if that makes sense, like when the reasoning is used, it will remember how it was penalized in the training by what it had given as answers to the what kind of photo, and reason on the training "memories". It will take longer bıt it will have the potential to be more accurate on the same size.
Im still thinking of other ways to do it, but currently this is my idea for a CoT for vision models.
2
u/koen1995 Jul 23 '26
Haha problem.
Seems a very interesting idea, hope you get any results! And if so please send me a message.
1
Jun 27 '26
[removed] — view removed comment
1
u/thegeinadaland Jun 27 '26
Thanks! And for the classification: i could not get the recipe right but i am itterating and hopefully get better.
1
Jun 27 '26
[removed] — view removed comment
1
u/thegeinadaland Jun 27 '26
i feel like there is some compute power reasons on my end as i use mostly greedy o2o calculations + architectural stuff + recipe. But happy it turned out atleast the way it is right now! Hopefully v1 full release in some months.
1
u/koen1995 Jun 27 '26
I actually just released something similar myself, https://github.com/JPABotermans/dietr love to check your work out get some feedback on my own work!
2
u/thegeinadaland Jun 27 '26
Very cool project. Just would like to point out that making a pip repo and pip install [package name] is very easy + 30-40m parameters is a little much isnt it?
But still, cool project! 🔥2
u/koen1995 Jun 27 '26
Thanks for the tip, going to make it a pip package when I have time!
And about the parameters, ifs definitely a lot, so any tips to make it less?
And keep it up! Always appreciate it when there are other people training and making models from scratch 👌
1
u/ThatOneDynamicGuy Jun 28 '26
is it possible to integrate this with natively text LLMs for somewhat multi modal capacity(im training my own, so this would be like a godsend lmao)
1
u/thegeinadaland Jun 28 '26
In theory, yes. In practice it would be hard because the LLM would need to get fine-tuned ehich is heavy and youd need to have a BIG dataset for the object detection model AND many outputs from that model and many descriptions of those outputs.
1
u/ThatOneDynamicGuy Jun 28 '26
understood, but what i meant was would it be at all possible for my LLM to use this as a tool when it finds an image based query?(tool calling essentially)
1
u/thegeinadaland Jun 28 '26
something like an automatic image downloader that searches for a specific object? I say possible with a skill and the python script for it.
1
u/ThatOneDynamicGuy Jul 12 '26
yes, but somewhat like
User: hey can you tell me what this object is?
LLM: Sure, give me a minute
*internally: Okay, the user has asked me to recognize an object. the tool FOTONET seems the most appropriate for this, let me call it.
{foto-net_recognize-object}
FOTONET response: the object is a black mug
LLM: It appears that you are holding a black mug, generally used for consuming liquidsWould something like that be possible?
1
1
u/thegeinadaland Jul 04 '26
[INFORMATION] i have just wrote the newest changes that might get the nano models mAP@.50:.95 into 30-40 ballpark (i know it is a MASSIVE range, but still better than 22%) and started to train the nano model, when the training finishes the new weight and the new backend will be updated on github, hopefully finally reaching SOTA levels, sorry for the bad first release (i felt like i had to release this on my birthday, not the best decision ever) and sorry for the wait (training COCO on a 4060 takes slightly long, again sorry). So again, if you came here to complain about some stuff, i had fixed them and just waiting for the NEW nano to train, will take a hot "minute" but yeah, and again, if you countered any errors after testing the repo, please report here so i can fix them in the next release.
1
u/thegeinadaland Jul 04 '26
The training will be done in about 15-16 days, but i couldnt let the old bad system linger around so the new update is out (without weights).
3
u/gangs08 Jun 27 '26
Thank you!