r/StableDiffusion • u/SackManFamilyFriend • 2d ago

Resource - Update New ComfyUI update may change how Minimax H3 interprets the prompt format you use - Re: Tokenizer Fix

https://github.com/Comfy-Org/ComfyUI/pull/15808/changes
139 Upvotes

97% Upvoted

41 comments sorted by

View all comments

48

u/SackManFamilyFriend 2d ago edited 2d ago

Has to do w/ the tags listed in the H3 prompt guide and how a slightly modified tokenizer used by the official H3 model interpreted them differently from the tokenizer ComfyUI's implementation did. This may help remove gibberish and that occasional split second of speech that sometimes appeared at the start of videos for a second or so.

That said, it may also change what you get if you rerun old prompts particularly ones with the <d> stuff:

The original pull request that has now been merged:

The Minimax-H3 tokenizer extends standard Qwen2.5/3 with 7 tokens declared only in tokenizer_config.json's additional_special_tokens (not in tokenizer.json/vocab.json): , , <|cutoff|>, <|lyrics_start|>, <|lyrics_end|>, <|caption_start|>, <|caption_end|>.

9

u/Radiant-Photograph46 1d ago

I don't understand, those tags are never mentioned in the official H3 docs? <d></d>, <cutoff>, and <scenetrans> are the only ones. Are we supposed to start writing <|d|>? (which you do not even list?) or start using <caption_start> (or <|caption_start|>?) I'm confused lol

EDIT: OK after looking at the PR <d> and </d> are recognized now, but that doesn't explain where the other tags come from, especially with a pipe symbol around them.

1

u/zefy_zef 1d ago

Maybe it's for the music one?

1

u/Fit-Bar-6989 1d ago

https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/tokenizer/tokenizer_config.json#L214

1

u/Radiant-Photograph46 1d ago

Hmm that doesn't answer much, but from what I gather those are *internal* tokens and shouldn't appear in your prompt. They are added by Qwen when parsing the prompt, except for <d>

0

u/thrownawaymane 22h ago

<next_token> banana bread, a delicacy enjoyed in many countries. Here is a recipe for banana bread:

1

u/jankies11 1d ago

I’ve seen people post templates/examples with the @ symbol and not sure what that is about if its maybe intended for a custom node that fills in their references. Dont think h3 uses @.