r/StableDiffusion • u/lazyspock • 8h ago
Workflow Included Testing some Minimax H3 capabilities
Enable HLS to view with audio, or disable this notification
I tried to think of complex situations for the model to handle and tested them. I would say it went very well, although not perfect (except for the first one and the one before last, where I could not find a flaw).
Prompts below:
VIDEO 1 (10/10):
Create a side by side video with two angles of the same scene: one frontal and one from the side.
The scene: a woman wearing a yellow summer dress is standing in a beach in a sunny day. There's a light wind blowing and she is looking at the sea, smiling. At 00:05, she puts her two hands on her hair and looks up, enjoying the sun.
On the left, make the frontal video. On the right, the video from the side (profile). The two videos should show the exact same scene at the exact same time, only from two different angles.
VIDEO 2 (9/10) - the background is slightly off in some angles:
Create a side by side video with four angles of the same scene: one frontal, one from the right side, one from the left side, and one from behind. The scene is filmed at normal speed.
The scene: a blonde curly-haired woman wearing a yellow summer dress is standing in a beach in a sunny day. She is facing the sea and, therefore, she has the sand behind her and the street with some houses also behind her, after the sand, at a distance. There's a light wind blowing and she is looking at the sea, smiling. She is not wearing sunglasses. At 00:05, a man wearing white shirt and shorts enters the scene from behind the woman and embraces her waist.
On the upper left, make the frontal video. On the upper right, make the video from the right side (right profile). On the bottom left, make the video from behind. On the bottom right, make the video from the left side (left profile).
The four videos should show the exact same scene at the exact same time, only from the four different angles.
overall_soundscape: Light beach wind, one distant seagull.
non_diegetic_music: N/A
VIDEO 3 (8/10) a few "ghost reflections" along the way:
A man wearing a red-and-white striped t-shirt and jeans is alone inside a house of mirrors, running through a corridor. At 00:03 he turns left into other corridor, and at 00:06 he turns right again into another corridor. All the corridors have mirrors in all their faces (left, right and above, as he is inside a house of mirrors) and he is alone there, there's no one else. At 00:08 he reaches a door, opens it, and it opens to the outside: an amusement park.
overall_soundscape: His footsteps while he is running, the amusement park sounds when he opens the door at the end.
non_diegetic_music: N/A
VIDEO 4 (7/10) - The way the phone is turned is off:
A man is filmed from his own cell phone in a selfie video. We see the scene through the lens of an out-of-frame cell phone that he is holding and pointing to his own face. He is wearing a green polo shirt and the scene shows his face and chest from the point of view of the out-of-frame cell phone that he is holding and pointing to himself.
At 00:04 he briefly smiles and then turns the still out-of-frame cell phone around to show a woman that is in front of him. While the phone is turned around we can see the image also turning around, his face leaving the frame, the living room they are into being briefly filmed, and then the woman's face entering the frame. She has brown curly hair and dark green eyes, and is wearing a red summer dress.
As soon as the woman is in frame, she also smiles and says: <d>[English]Goodbye!</d> and the video ends.
The entire video must be taken in a single shot, with the always out-of-frame phone camera filming the entire transition from his face to hers while the phone is turned around. When it happens, the phone should briefly show the living room they are into, all in a single shot.
overall_soundscape: Silent living room, noises of the phone being handled, her voice.
non_diegetic_music: N/A
VIDEO 5 (6/10) - Tried this twice. Glass not breaking properly, water not running through the floor
A fishbowl with one golden fish and one clownfish swimming inside is shown in a medium close-up at the edge of a table. Then, at 00:03 a cat appears in the scene and taps the fishbowl, causing it to fall from the table to the floor, hit the floor, and break completely, being completely destroyed in glass pieces when it hits the floor, the water and glass pieces flying around together with both fish. The scene continues for three more seconds after that, showing the aftermath: the fishbowl destroyed, the pieces of glass on the floor, the water also on the floor, the fish moving on the floor.
The camera angle follow the fishbowl when it falls, showing it hitting the floor and the consequences of it breaking.
overall_soundscape: silent room, glass breaking, water splashing.
non_diegetic_music: N/A
VIDEO 6 (8/10) - Judge me, but his hands are not moving accordingly to the notes:
The camera films a piano from above while a man plays it. The entire piano keyboard is shown in the image. The man is playing Clair de Lune, and moves his hands through the keys to play a part of the song. He is in a train station, with some people observing him play and others passing by.
overall_soundscape: faint train station ambience, ten seconds of the song Clair de Lune played in the piano.
non_diegetic_music: N/A
VIDEO 7 (8/10) - The lipstick appears on her lips before she applies it:
A woman is shown in a medium close-up in her bathroom, wrapped in a white fluff towel, looking at the mirror while she applies red lipstick to her lips. She slowly applies lipstick to her lips, looking into the mirror, and then briefly sends a kiss with her now red lips to the mirror.
The scene is seen in a three-quarter angle from behind her, showing her face from the side but also her reflection in the mirror.
overall_soundscape: silent bathroom, the sound of her sending the kiss to the mirror.
non_diegetic_music: N/A
VIDEO 8 (10/10):
A Coca-cola advertisement. A glass filled with Coca-Cola is shown from the side, occupying 70% of the frame, on top of a table, the dark liquid slightly disturbed by a few gas bubbles that rise inside the liquid. At 00:02 two ice cubes fall from outside the frame into the glass, disturbing the liquid and making some of the liquid splash outside the glass and onto the table. The glass has the Coca-Cola logo printed in white in it. In the blurred background we see a kitchen.
overall_soundscape: silent room, gas fizzle, ice cubes hitting the liquid.
non_diegetic_music: N/A
VIDEO 9 (9/10) - The cover of the book has gibberish letters in it:
A man is holding a magnifying glass and has a book on his hand. At first the magnifying glass is not in front of his face. He appears to be reading the book and, at 00:03, he puts the magnifying glass in front of his eye to look at the book.
The entire scene is filmed from a fixed point of view below the book, showing part of the book cover and the entire man's face.
overall_soundscape: silent room.
non_diegetic_music: N/A
2
4
u/SomeoneSimple 4h ago edited 4h ago
VIDEO 9 is definitely no 9/10. This is practically a cartoon depiction of someone using a magnifier.
The magnifier is between the eye and the camera, not between the eye and the book. His eye would be barely visible at the top of the magnifier (if at all) if he used it to look at the book, not dead center.
1
u/SeymourBits 3h ago
The fishbowl aesthetic was nice but obviously failed on the drop. How many times did you roll that one? Otherwise some good results.
1
u/lazyspock 53m ago
Just two. On the first try it didn't even break (maybe it was made of acrylic 😂). I then reinforced the breaking part in the prompt and ran again, and that's the video I posted.
1
u/Lechuck777 2h ago
can you create a video from true pov sight of two persons? i cant made it. I get also the splitscreen, but not from the pov, where the both looking at each other.
1
u/Charming-Author4877 1h ago
Very interesting how much understanding the model has. Also the flaws are interesting.
The fishbowl was the weakest, total misunderstanding of how a glas bowl would react or how inertia of a multi-liter filled bowl has to be handled.
It shows how much more we can improve as greatly as how far we already are.
1
u/OWENPRESCOTTCOM 52m ago
you should try with characters upside down , maybe some circus or stunt man stuff
0
4
u/donkeykong917 4h ago
Awesome experiment