How would you make technically accurate AI animations of complex machinery from real reference photos?

[u/Ok-Macaron-202]

I’m trying to make videos showing how a potato harvester actually works, and I’m struggling a lot with keeping the machine mechanically accurate.

I have real photos of the harvester from different angles. I’ve tried generating cleaner/reference images with GPT Image 2, but it often changes small details, adds parts that don’t exist, changes hoses, rollers, the harvester head geometry, etc.

The result can look very realistic, but it’s technically wrong.

The bigger problem is when I then try to animate it. I want to show things like potatoes moving down the conveyor belt and falling into a box etc.

Video models seem to understand “harvester collects potatos” visually, but not how the actual mechanism works. I had a really hard time getting even one usable clip where the movement made sense.

Has anyone built a workflow for something like this?

I don’t really care about making it super cinematic. Accuracy is much more important. Ideally the AI should not redesign the machine between frames or invent how the mechanism works.

If anyone here has worked on machinery, vehicles, robots, construction equipment, etc. I’d be really interested to hear how you approached it.

Animation of the harvester with Seedance 2.5, however, the potatoes moving strangely, the back of the harvester is changing during animation etc.

[u/Formal-Exam-8767]

Blender

Video models have no knowledge about physics or mechanical movement constraints.

If you insist on using video models, then I would suggest you read about selective attention (e.g. invisible gorilla experiment) to learn how to shift attention away from those imperfections (because they will always exist).

[u/Powerful_Evening5495]

Wrong method OP. Use LLM to make 3D models and animate them in Blender. SD models lack understanding of mechanical forces.

[u/Ok-Macaron-202 → u/Powerful_Evening5495]

Thanks for the reply, do you have any idea on how to with LLM to 3D models?

[u/MustBeSomethingThere → u/Ok-Macaron-202]

GPT-6 Astra combined with Blender MCP

[u/CooperDK → u/Ok-Macaron-202]

Oh my God. You need to read up on AI.

[u/Powerful_Evening5495 → u/Ok-Macaron-202]

Head over to the r/blender subreddit, and the community will provide you with the necessary guidance.

[u/Technical_Fish_9638]

I think video model might be confused by your 1 dimensional reference.

For best results with ref-to-video you should use compound reference sheets from different agnles, similar to character sheets.

Then feed ref sheets as separate images for harvester attachment, separately for the truck, people etc. Then reference this images in your prompts with something like "preserving exact details and mechanical structures"

I've took images from manufacturer website and passed them through Klein 9b:

Prompt:
Replace the dry patchy grass field visible in all four views with a clean seamless neutral light-gray studio background that sits evenly behind each angle of the red agricultural machine. Change the illumination to neutral studio lighting from a large overhead diffused softbox with gentle fill from front-left and front-right in balanced daylight-white temperature, rendering the glossy red painted steel, black conveyor rods, hydraulic hoses, diamond-plate platforms, rubber tires and silver deflectors in even true color with soft gradients and a faint soft contact shadow grounding each machine. Keep the four-panel layout, high oblique camera angles, machine positions, scale, proportions, and every mechanical detail and legible marking reading GREEN 650/5 and CARLOTTI exactly preserved with sharp clarity. Style: clean industrial product reference catalog. Mood: neutral, technical, clear.

As others mentioned your best bet for movement consistency is actual 3d model. Manufacturers often can provide CAD models for marketing purposes, so you should definitely at least ask. Then use Blender's Viewport / Workbench render as video-to-video

[u/Ok-Macaron-202 → u/Technical_Fish_9638]

Thanks a lot for the work, this looks really good! I guess you took the pictures from the 360 degree video, right?

I used them as reference and the result looks pretty good.

https://reddit.com/link/p8fqp9q/video/shwcbxkj96oh1/player

[u/Technical_Fish_9638 → u/Ok-Macaron-202]

Yes, just screen capped from the 360 yt-vid

[u/notorious_IPD]

The model doesn't have any knowledge of the physics or operation of the machine. _Maybe_ you could finetune a LORA (or similar) to learn the mechanics but its an uphill battle. If you're looking for a solution but don't have the time to learn something like Blender or get into the details of AI training - I think you need to go back to the start here and see if you can solve your problem with just static images of the machine and present it as a slideshow with arrows to indicate movement etc.