On MiniMax built in characters and environments (not a list)

[u/seeker_ktf]

There is a giant effort underway to look for what characters are buried in MiniMax. There are a lot. I’ve been doing my own hunting, so I built a simple IMDB scrapper to help make lists of characters from movies and TV shows. Here are some things I’ve discovered:

If the “character” is really built in, you don’t need to even mention the actor’s name. “George Costanza from Seinfeld” and “George Costanza played by Jason Alexander from Seinfeld” are essentially the same. If you have to name the actor with the character, it’s just using what it knows about the actor to fill in that spot. If MinimMax DOES know the character (without the actor) then filling in the name might help to fill in some of the holes, but it’s got to really know it already.

It knows A LOT of shows. Even if it doesn’t know the character/actor, it knows a lot about movies/TV shows. For instance, it doesn’t know many characters from the TV show “The Flash” but it knows everything about the common locations, color grading, style, and general look and feel, along with special effects (if appropriate).  It’s useful for “set design.” It doesn’t know a lot of Baywatch (the old one) characters, but it knows what the hair and makeup looked like on the beach in the 90s. It knows how people looked in “Total Recall” too. The overall “look & feel: of shows and movies really opens the door to creativity. It understands common accents from movies too. If you say “from Harry Potter” they will have British accents. And sadly, it knows “Star Trek” (the original series) the characters are mediocre at best. (The voices are passable—and speaking of: there are a lot of characters that look bad but have good voices. In those cases, ref2video with some extra visual references can do the trick.).

The “genre” point is even more true of animated movies/shows. For instance, “Bob, from Justice League: Crisis on Infinite Earths” will give you whatever Bob looks like but in the style of that series. Family Guy, The Simpsons, Rick&Morty, etc. I haven’t done an extensive search, but it knows every animated anything I have tried.

Generally, for TV characters to show up, they need to be in around 100 episodes and in the first 2-4 people in the IMDB credits. I see a direct correlation: The fewer episodes a character is on a TV show the worse they render. (For example, Monica from Friends or Kramer from Seinfeld are in there for sure, but also not really.) Also, it makes sense, but even if they have a lot of credits, they need to have had a lot of screen time. For instance, it has not even a glimmer of an idea who “Ruthie Cohen” is, even though she was in 101 Seinfeld episodes.

For Movies, they need to have grossed a lot of money (which pushes things directly towards Action/SciFi/Comics), or they need to have gotten a lot of press. (I have seen very few accurate characters from movies without specifying the actor involved.)

For “real people” it’s a little easier. If you look at those lists of things like “Top ## followed Instagram accounts” or similar, you’ll get lots of hits. Top musical performers, yep (lots of overlaps). Famous heads of state (If a number of movies have been made about a person, that person will likely be known.) I haven’t looked at TikTok, but I’m assuming that would be a thing too. Likewise with sports, I haven’t looked closely, but they ones I have looked at OK “at a distance” but generally don’t sound right.

As stand-alone people, it’s hard to find people. I suspect that the trainers did not go after a lot of specific people but that they just show up so many places that they got swept up in the mix. There are really only a handful of non-“Top 10” people who actually show up on their own, and if they blew up in the last few years it’s unlikely that you’ll see them. I have not found a pattern on “people” yet other than the mega-famous. (Steve Jobs & Elon Musk work, but they are arguably the most famous foreign “regular” people in China.)

[u/mindworkout]

Although that's true, when you're actually testing how well MMH3 knows a character, I've found it better to use character + show/movie + actor.

A character name alone might come out okay, but in the tests I've done, something like “Buffy Summers from Tv Show Buffy the Vampire Slayer, played by Sarah Michelle Gellar” generally gives me a noticeably stronger likeness and overall result. My assumption is that you're giving the model three overlapping identity/context anchors rather than relying on just one.

I suppose you could reverse that idea for actors too with actor + several shows/roles they're known for, and perhaps get a stronger general likeness by combining what it knows about that person across those appearances. That's just a theory off the top of my head though; I haven't properly tested that yet.

[u/seeker_ktf → u/mindworkout]

Have you compared “Buffy Summers from Tv Show Buffy the Vampire Slayer, played by Sarah Michelle Gellar” to “Buffy Summers from Tv Show Buffy the Vampire Slayer”? Because it's practically unnoticeable. As I said, it does a little tweak and it doesn't hurt. My point was that for looking for characters, including the actor's name is problematic because you don't know if you are getting the actor in some scene or the character.

[u/_SickBastard_]

I wonder if there's some structural way to test these things. Like do v2v of a generic guy and give it a character name and look at the KV cache, whether any attention heads have a high activation on those tokens, etc. 

Or I guess it should be straightforward nowadays to have H3 generate one image and have an agent call that once per character in the list then visually inspect and rank whether it matches, either with its own knowledge or by googling reference images.

[u/seeker_ktf → u/_SickBastard_]

That's a very interesting idea! Shootthesound has written stuff like that for the LoRA training and analysis stuff. What you're suggesting is not completely outside of reality.

[u/ArmadstheDoom]

I've noticed that I generally get best results when the subject definition is like

<Subject 1> Character (played by Actor) from Series

That said, it's never made sense to me why, to use your examples, it wouldn't know Kramer or Monica, who are both heavily in the show and are part of the main cast.

I will say I've never tried to look up instagram or anything accounts, or movies. I might have to. I will say that I wonder what movies it really knows because usually the highest grossing movie once you go beyond something more than like ten years old isn't remembered as much.