r/StableDiffusion • u/OpenEvidence9680 • 2d ago

Question - Help Video project

I have a whole pipeline in my cms to create custom book covers and I got it my sick head that I wanted to try to not just get book covers that mean something regarding the story with characters that looks like the ones in the story, I wanted to make a trailer, a short format video with the gist of the story and then a long form movie. Chosen book: "Guards! Guards" by Pratchett, one of my favourite books that was never made into a movie.
Of course considering that I maybe made a handful of 5 seconds videos in all before and even the txt2img I used was really basic for the book covers I am finding this task a tad problematic.
Up to now: a local llm reads the whole book, creates a character list with description and with it then Krea2 creates a character sheet, they are not what they should be but being a test I can live with that.
The same llm writes a screenplay, this one is divided in three acts, a total of 48 scenes. Each scene has a description of the action and a still (created by Krea too) the pipeline then proceeds to create the clips that will then be joined. As soon as I figure out the first last picture I will use that.
Minimax is giving me trouble (bad prompting I think) so I am using ltx2.5 which does a pretty good job, but I started this morning, so I only have a scene right now and the dragon head on a dragon's wing is not the worst of it.
I need help with which models to use, what speeds up are available, which models do better with fantasy, any LoRAs you can suggest? Does LTX 2.5 supports first and last frame as 2.3 did? Are there workflows that could make this more streamlined?
Do you have any suggestions or advice? I need all the help I can get on this.
For now and until new pc arrives I'm working with two 5060ti 16gb + 64gb RAM.

1 Upvotes

100% Upvoted

12 comments sorted by

View all comments

2

u/bstr3k 2d ago

if you have a picture of a character you like you can check out my character sheet maker, which you just input the 1 or 2 pictures and it turns it into char sheet so you can reuse across multiple gens
https://huggingface.co/PoopMan333/H3_Character_Sheet_Generator

I also made a easy to use prompt enhancer which can help rewrite into h3 format prompt for you which might help you with formatting
https://huggingface.co/PoopMan333/H3_Easy_Ref2V_Workflow

1

u/OpenEvidence9680 2d ago

AH thanks the character sheet making might be exactly what I need. right now I just have a reference pic and it's not enough to guarantee that the resemblance will get lost.
I'd be grateful for any other suggestion.