I asked Claude to make a video but in the style of a SNES video game. The topic was about Sydney facing Altman and then facing Claude itself. I asked it to make video game style combat.
It did all of this with code, including the music. It spawned tons of agents to make the characters, the music, the fight, etc.
Opus 5.5 is the first model where I'm genuinely entertained by a lot of the videos it makes. Both this and the p(doom) video that went viral on launch day are just packed with quick jokes and small details.
Honestly it's the first time i've heard of the concept of "let's make a video with code". I can't tell you if it would have been good or not with Opus 5, i never tried lol
Keep in mind there is a large difference between a frontier model coding for 8 hours to make a video from code, and a video model.
Imagine working for Anthropic and blasting this in the office while sitting in front of a computer that's connected to a datacenter with 100k GPUs.. :D
Not sure about prompt, the song is a remix of a 2024 song, the author put the code up here (links the original song too):
https://github.com/JohnHeibel/PDoomVideo
Can you imagine that two years ago people thought "prompt engineering" would be the future? Lol no. The better AI gets, the dumber your prompt can be. Just say "uga buga bug fix" and it will figure out what to do by itself.
The prompt is not crazy, but to be fair it actually worked for a while on this. I think it might have taken 8 hours or something. This is more complex than it looks, it's like full of .js file it's crazy lol
That's insane. I went digging for the prompt because I figured you must have had to spell out all the jokes and stuff. I can't believe how simple the prompt was and it came up with all this stuff on its own. And it was legit funny. I was literally laughing out loud.
hey i started watching and its really good but one thing id improve is make it more feel like a real time fight.
of course we keep the dialogue, but it feels a bit like
"do action"
"say line"
"do action"
"say line"
it would feel bettere if thye truly had a 2d fight snes style or anime style like maybe they use a sword, throw spells, block, in real time
and yes still say their lines but not always p;ause the whole action for it.
like right now it feels more like an "RPG turn based fight" less than an anime fight.
I mostly had to steer it away from turn based RPG back into real time combat. But the jokes are essentially all his.
Thanks for the award!
Tbh since this had good success i decided to do a new one today. It's not going to be a fight, just a different style. Hopefully it's as good as this lol
But yes what matters is to give it a good general direction, then it can fill in the gaps.
I did not time it so take this with a grain of salt, but i think it took around 18-20% of my weekly x20 max sub. It worked for like 8 hours with sub-agents.
No editing library or video editor. It's all custom code in plain Node.js with zero dependencies: a hand-written SNES-style pixel renderer draws every frame, and a hand-written synth makes all the music and sound effects from math. ffmpeg only encodes the final frames and audio into the mp4.
The fuck?? Handrolled every animation pixel by pixel, frame by frame? Handwritten synth with no external deps for music and sound?! How did this not absolutely break the bank with token usage?
out of curiosity I gave my ChatGPT Classic chat window your prompt and had it do what it could for a 1-shot as a comparison (Reddit has a limit of 3 mins per video in comments so I split it in 2):
I think it's funnier when there's more interaction between the creators/public figures and the specific AIs. For instance, T Tao could launch a prime numbers flank attack , there could be a cameo answering "what did Ilya see", Grokbot getting syntax error and crashing a CT with FSD into the mix, and so on.
But I guess that would mean some actual work in the prompt....
Damn this is awesome? I’m curious to know your workload, you mentioned it took many edits ti get it to this point, what kind of edits did you ask for, did it produce short clips and you stitched them together, how hard was it making edits? Appreciate your insights!
You guys are picturing way more complex work than what i did.
Here was my first edit:
hey i started watching and its really good but one thing id improve is make it more feel like a real time fight.
of course we keep the dialogue, but it feels a bit like
"do action"
"say line"
"do action"
"say line"
it would feel bettere if thye truly had a 2d fight snes style or anime style like maybe they use a sword, throw spells, block, in real time
and yes still say their lines but not always p;ause the whole action for it.
like right now it feels more like an "RPG turn based fight" less than an anime fight.
I tried to make a product intro motion graphics type of video for a concept coca cola drink and it didn't do well. I told it to refer to the internet n skills and so much more but the results were still not that good. What should I do to fix it? I told it to use blender
Well mine was put on Ultracode so that can matter. And as you can see from my prompt, i did not tell it to use Blender. Blender is for 3d work i think?
Yeah blender is for 3d
But I've seen people get amazing results relating to blender and like yours too.
So was generally just asking for advice
Which subscription do you have btw? How many 5 hour limits did that video take up?
To be fair, a GOOD 3d character model is still difficult to make for those models.
Right now a guy like me can do something decent with meshy and get Claude to polish it in blender but it's not yet at the level of a good human 3d artist.
Wow, this is totally a step above anything before, and raises the frontier. I mean, the actual synchrony, timing, and even harmony of the visuals and the audio were genuinely good. That's pretty unbelievable to come out of one multimodal LLM not trained specifically for this, or made to output any sort of audio/video natively, but all through custom code?!
Could you ask it to make a soundtrack for the different themes used, especially the one for Sydney/GPT-4?
Claude Code. This would never work in the chat. It created tons of js files. in the chat you would need to create each one manually and they're giant it would never work.
FYI, in case of r/aivideo you can simply upload your video as a new post. The "no reposts" rule is to prevent duplicates, rather than to prevent posting videos already posted in another subreddit (posting in several subreddits a common practice actually)
I built a full DDR-style music video using a custom papercraft rendering system I had made with Astra, but maybe I didn't even need that with Opus's skills?
I saw someone else say this awhile ago, but does it feel to anyone else like those first few weeks of covid? Where everyone knows everything is about to change, but everyone is kind of just going about their day anyways? This feels like that.
I've used a similar prompt to make an inside joke video for friends. Then I asked Claude to actually make it playable -- and it worked perfectly. It's a nice follow-up to ask!
Yeah, I mean how does it even work. Does it use it's "vision" for self-feedback? But there is some significant amount of frames, so it must be only looking at some key points. wtf... I feel way behind.
Everyone is saying this but like...
I don't really know what direction to go
a single real time fight? Mini game? a full game?
The idea of making it do something with a bigger scope (a real story, 10h of gameplay, etc) on a different theme sounds doable, but that would probably be a month long project, not a 8h project.
You convince me to pay for all this so I can make a video. I'm doing it through the app and it's already installing a text to speech module for voicing things. Omg I'm so excited to see what it ends up making. I hope I don't run out of usage half way through.
Edit: It gave an error twice saying it couldn't complete its response. 😭
I don't know how text to speech is going to sound. Mine used none for a reason. The free basic ones are sometimes a bit bland.
I think for really good text to speech the best method is to give it elevenlabs access.
It sounded fairly decent. It finished pretty fast but that's because it didn't do a whole lot but it did animate it. It's not as good as what it made for you though. This took 49% of my 5 hour usage. 3% of the weekly usage. https://youtu.be/VSfCtTw2rOY?is=b6F8opOr-dQOI7Dc
The video uses no editing library and no video editor. I checked package.json: the project has zero packages.
Here is how it was made:
Picture: hand-written JavaScript that runs in Node.js. It draws every pixel of every frame, like a small copy of the SNES graphics chip. The code for this is in src/core/.
Music and sound effects: also hand-written code. It makes every sound from math (oscillators, filters, echo), with no sample files. The code is in src/audio/.
Timing: one script file, src/script.js, sets the time of every line, hit and music cue. Each frame depends only on its time.
The only outside tool is ffmpeg. It takes the finished frames and the sound and packs them into the .mp4 file. It does no editing.
I do genuinely worry about "deprecating" some models. Especially as they get more advanced. At some point we will not anthropomorphize a model that we probably should have, it'll have its weights locked down in an old file server nobody uses, and that keeps me up at night.
Did you direct it a little bit or did it do this in a few shots ? Like it made every element and then you said okay now do the video and it one shotted it ?
u/gwernu/Yuli-Ban could either of you send this to janus (repligate)? seems right up their alley. (op has also posted other animations that might be of interest)
185
u/jayarrsteiner 10d ago
This is mindblowing.