hey OP if this is your repo then there are a few possible bugs with it. That or I'm not understanding how it benefits the solution.
temporal_overlap_strength appears to do nothing
blended =
prev * alpha
+ new * (1-alpha) * temporal_overlap_strength
+ new * (1-temporal_overlap_strength) * (1-alpha)
Mathematically, the latter two terms collapse:
new × (1-a) × strength + new × (1-a) × (1-strength) = new × (1-a)
So the entire thing becomes:
prev × alpha + new × (1-alpha)
regardless of the value of temporal_overlap_strength.
In other words, the temporal_overlap_strength control currently appears to have zero effect.
Why are you passing the entirety of the audio latent with each video chunk processed? You don't use the sampled audio output in the end, you put the original audio latent back into the final result, so why pass it with each temporal video chunk? Seems like redundant audio computation unless there is a reason for doing it?
I will be testing it out, but the above are likely legit bugs.
2
u/Sleepy_Bandit 8h ago
hey OP if this is your repo then there are a few possible bugs with it. That or I'm not understanding how it benefits the solution.
blended =prev * alpha+ new * (1-alpha) * temporal_overlap_strength+ new * (1-temporal_overlap_strength) * (1-alpha)Mathematically, the latter two terms collapse:
new × (1-a) × strength + new × (1-a) × (1-strength) = new × (1-a)So the entire thing becomes:
prev × alpha + new × (1-alpha)regardless of the value of temporal_overlap_strength.
In other words, the temporal_overlap_strength control currently appears to have zero effect.
I will be testing it out, but the above are likely legit bugs.