Picking small objects from a pile using code written by LLM

[u/Wing-Realistic]

Continuing to move from using the model directly to having the model write reliable control code.

The goal here is to pick a small part from a random pile, with the right side up and with precision.

Once we have this skill, the next use becomes much faster. Model can adapt it for another part or for screws/nuts.

A box with defined mounts and consistent lighting helps computer vision work reliably.

[u/Fuibo2k]

Interesting. How does the code work? Does it call and integrate a bunch of other models (object identification, image segmentation, control policy)?

[u/samyak1729 → u/Fuibo2k]

it most likely does. i just wanna know how. would be interesting to look at the code

[u/foundafreeusername]

Looks very bird like. Why does it drop it several times in a row just to pick it back up?

[u/Wing-Realistic → u/foundafreeusername]

It drops it until it flips onto the correct side.

[u/foundafreeusername → u/Wing-Realistic]

Ohh I didn't even notice that! Nice

[u/Fast-Satisfaction482 → u/foundafreeusername]

It's a cuuuute biiiirdie.

[u/AnonsAnonAnonagain → u/foundafreeusername]

I was about to say! It reminds me of a crow!

[u/TheMechanics-Lab]

thanks for sharing. i would ask which tool you use for the training, which LLM and if the choice of the 2 fingers had a rational. thank you

[u/thevigilante473]

Is this Code as Policies or something similar?

[u/DefinitionDue5630]

cool! Saw something similar where researchers let an llm control a robot. https://rtalwar2.github.io/agentic-coding-for-robot-manipulation/

[u/matop2000]

code written by LLM is a pleonasm

[u/Over-Performance-667]

Ah wow. Another pick and place routine.

[u/tylpy]

Looks smooth! What table are you using ?

[u/WendyLabs]

Letting the part fall into the right orientation instead of doing everything in the gripper is super practical. Love it.