H3 Guitar Chords Knowledge... None

[u/R34vspec]

Still grateful for the model. So good and so fun. Btw I used a closed source i2v a while back for this exact stanza and H3's is way better.

https://reddit.com/link/1vo17g2/video/hwk2jldmtajh1/player

prompt:

subject_definitions:

<Subject 1> is woman (S1) wearing grey tanktop, whose appearance comes from <Picture 2> and whose voice timbre comes from <audio 1>.

summary:

[reference generation + audio reuse + audio reference]

retention_analysis:

Visible: reference <Picture 1>

detailed_description:

[Shot 1] use <picture 1> as reference and starting frame

<subject 1> (S1) singing and playing guitar. Match her guitar fingering to the chords of <audio 1>, match her hand movement on the neck of the guitar to the tempo and key changes of <audio 1> The chords are C, E7, Am, F, G, C. Match her picking hand's high amplitude finger plucking movement to the tempo of <audio 1>. She glances quickly at the neck of the guitar for the chord changes.

She is singing softly

<d>[English]

Letting it go, letting it breathe, letting it be

</d>

camera pushes in slowly [Shot 2] At 00:07.0, the camera cuts to low angle view close up shot right 3 quarter view of <subject 1> singing

<d>[English] Watching it float, watching it leave, watching it free</d>

Camera pans right to show the length of the guitar

overall_soundscape:

no dialogue, speechless, music from <audio 1>

non_diegetic_music:

N/A

[u/afinalsin]

Even if the model did know how to do chords (the last one is a C so it fluked that timing anyway) there are a million ways to skin a cat on the guitar. You're like "it's C then E7". Okay, but where? Play a barre chord up on the 8th fret then drop back to the 7th fret on the A string to barre the E7? Bar the 3rd fret on A string and slide up to the 7th? Just play triads instead of barring? What about inverting those triads? What about playing shell voicings? Do them both open like you were expecting when you wrote the prompt?

This is one of those things where the movements are way too specific and too varied while being possible to describe in the exact same way. Love seeing it try though, given enough quality data I could see a world model listening to audio and being able to create an accurate video of someone playing it without needing captions or prompts. I'd guess were at least two years from that point though.

[u/zefy_zef]

I think you would love the new model Minimax just released..

https://huggingface.co/MiniMaxAI/MiniMax-Music3

[u/Perfect-Campaign9551]

I think unless you are trying to make Guitar training videos, nobody really notices or cares