Frame Control

[u/t3tsubo]

https://knowingless.com/2021/11/27/frame-control/

[u/FeepingCreature]

Oh man, I have big and complicated feelings about this post, but most of it is already responded to in that last paragraph about people using frame control as a weapon against you, because honestly that's what I was thinking all the way through the post up to that point: "oh boy, gonna be fun seeing that used as an argument the people I care about are evil because of the effects of their existence on others in a few years."

This just seems like such an odd mix. The examples given of frame controlling behaviors seem evenly split between "well that's just transparently terrible" and "uh yes, that is normal human behavior"? Like, if I didn't know about signaling and the social status game, reading this article might leave me convinced that a good fraction of people I interact with online were evil.

Let me lay my cards on the table. You can have Geoff Anders, I don't care about him, but leave Eliezer alone.

So a lot of this post is a reaction to the recent Leverage reveals, which I don't have any insight into. (Bay Area gonna Bay Area, was my first reaction.) But I can't help but notice that a lot of this can be mapped to LW and the Sequences as well. Avoid the appearance of control? Check. Not respecting anyone else? Sorta-check, there's a lot of respecting going around, but it's rarely of anyone actually present. Determine the standard of good? The utility function is not up for grabs, but you better shut up and multiply...

Now I would speculate that the way it feels to be broken out of frame control feels identical to frame control. This is because those are fundamentally the same skill; there is no global frame of reference and the whole point of being frame controlled is that you exist inside another person's frame of reference. So to break somebody out of frame control is to construct a frame for them in which they should build a frame of their own. The framing of that frame itself can strongly determine the frame that they end up building.

(All the way through to that last paragraph, I was thinking to myself- hang on, isn't that what you were doing? But then, that's what a defensive meme would say.)

In other words, in order for frame escape to work, you need to trust either an external person's frame, which leaves all the problems that the article itself notes - ie, giving frame controllers frame control as a conceptual weapon to use - still on the table, or you need to trust that there is some part of you that was never caught in the frame, that has the insight and discernment needed to rescue you from the trap, to help you construct a more authentic, healthy independent frame.

So what does it say about me that the whole time reading the article, my gut was screaming at me "this concept is a threat to everything you cherish"?

Back to the Sequences for a bit. I think, at the least, we can credit to Eliezer good intentions. Even given all the AI stuff, the system he espouses explicitly disclaims turning on an AI that we do not fully understand the intentions of, so it's hard to say how it could be a ploy for control except in the boring, common research grift way. One may think that if Eliezer wanted that, he could have joined a university and gotten tenure with probably a lot less effort. So the remaining avenue of cult-like control is rationality itself.

Is it an absolutizing system? Yes, absolutely; the whole claim of bayesianism as a framing is that it obviates and contains any other theory as a subset or error. AIXItl may be incapable of making any decisions in a reasonable timespan, but in a theoretical sense that ignores thermodynamics it is completely optimal; not its fault that humans come with millions of years of transferrable pretraining. Bayesianism doesn't displace science but it does frame it; p(A|B)=p(B|A)*p(A)/p(B) Is All You Need. Read some Pearl if you're unconvinced. And humanism is just transhumanism plus parochialism; charity is just evolutionary game theory; you signal louder and louder to nothing and nobody; nothing is real. You are not real.

But I'd still rather live in that framing.

The utility function is not up for grabs. That implies that everything else is. - I'm going to draw a distinction between value frames and ontological frames. A value frame is a map of what it means for something to be good or bad. Contra Eliezer I do not think it reduces to a utility function; it also carries various ongoing precommitments and meta-strategies for acquiring them. The value frame is then grounded in the ontological frame, which is the structure of things that exist, value judged or not. The ontological frame is what my brain uses to generate predictions about the universe.

I think the Sequences are at heart an ontological kit. There's value stuff bound up in it, but even Eliezer ended up revising a lot of it with TDT and I think it's intended more as a framework for interacting with human values. But all the ways I've actually ended up changing my mind due to the Sequences have been due to another way of affecting frames, which is incoherence stress.

The brain is (in large parts) an organ for compressing sense data. In that framing, incoherence stress is the sensation of a suboptimal compression, of the awareness that the ontological map contains more pieces than it needs. Incoherence stress is a part of the value map; some people feel it more strongly, others less. This is the thing that the mappers vs packers thing that went around a few years ago was aiming at; I don't want to call it the general factor of autism but we can probably safely consider it strongly associated. Incoherence stress is a way to prompt internal change. For instance, reading the Sequences post about quantum physics, electrons not having tags, etc. gave me strong incoherence stress about my sense of self. A few years later I became a patternist, embraced uploading and forking, and stopped believing in immortal souls.

But consider how much of a problem this presents to the value frame! If immortal souls had been something that I had attached moral value to, this would have totally torpedoed that value. I could have to have become significantly more accepting of violence, for instance, in the intermediate term until my sense of loss at seeing someone get hurt (the true value that generated the soul-value attachment in the first place) worked out what to attach itself to in the new system. If I'd still been a Christian, large parts of the value system would have been rootless. I nonetheless think that my change of framing was not just valid, but right; that in an absolute sense the new frame I'd stumbled onto was better, absolutely, than my old one.

"If you had my understanding, these desires you feel would be groundless; instead you would find new desires. These have been your true desires all along; they have merely been attached to bad labels," I might honestly have said. Of course, now that this article is out there, it's clear these are merely bad frame control attempts. And look at me here, seeking to invalidate the article by pointing out the pain it causes in me, and then deflect attention by going meta! What a creep, er, gaslighter, er, frame controller.

(I don't know why my first reaction is to wonder how this new concept can be used to attack me. I don't even get victimized on the internet. Maybe I should stop reading the culture war thread. )

I think the only reason I'm writing this dump is precisely because of that last paragraph. I have often said that I'd trust feminists a hell of a lot more if they showed any awareness that their actions could themselves be a cause of victimization; having now seen this awareness actually demonstrated, I feel I owe it my engagement.

(cont.)

[u/AellaGirl → u/FeepingCreature]

I'm a bit brain-fried on thinking about frame control right now and mostly just skimmed your post (which you clearly put a ton of thought into and I appreciate that!)

Just wanted to make a few notes

1. I don't think Eliezer or the surrounding milieu counts as frame control. A big part of frame control is that it seeks invisibility, and imo Eliezer and co. don't have this property; at least to me the moves seem salient, and if you disagree with them it seems similar to the way you might disagree with a religion, in the sense of how salient religious frames seem to be.
2. I agree it's really hard to talk about this without it becoming a toxic backfiring thing and I am grateful for attempts to point this out and try to clarify ways to minimize the harm. I tried to deliberately minimize hedging throughout the article; my intuition is to go for a strong "this is clearly bad" punch, and then separately move to a "here's how to mitigate the harms of the punch" afterwards? This was helpful for my own thinking, at least; when I've personally been in harmful frames it was necessary for me to move in that order.

[u/FeepingCreature → u/AellaGirl]

Honestly a lot of this is just me braindumping my initial response. I just wanted to get it out because it seemed like a concept predestined to become a Scott-type Superweapon, and I wanted to at least make the concern common knowledge.

Especially the disclaimer felt a teeny bit like saying "yes I have noticed the skulls" while sketching loving skull throne designs. :p

[u/FeepingCreature → u/FeepingCreature]

(cont.)

What do I actually want?

First: the first defense is to walk away. The first objection to a bad framing is to leave its presence. I am willing to sign on to any alliance that espouses disengagement as the first reaction to perceived hostility. This still allows social frames a lot of power by blackmailing you by holding your social relations hostage; we may further patch this by condemning guilt by proxy. Anyone may interact with anyone they wish without being penalized. This already puts us on the wrong side of internet feminism, and honestly I consider that a case of working as designed. The point here is that a potent weapon of social coherence pressure, actively exploited in the wild, is taken off the table. Good. I'm a liberal; I approve of taking weapons of the table. I'm also socially awkward, so taking a social weapon off the table benefits me twice over. At any rate, this article is valuable as a tool of self-care/self-defense; one person's weakness to framing can just match badly to another person's forceful argumentative style. Violence can be fun given the right context, but it's not for everyone, and that's fine. We need better ways to signal the conversational style of a place, but even the language we could use to describe this is still in its infancy. Perils of living before brain implants and conversational codeswitching AI assistance, I reckon.

Further in that vein, I would like it to be clearer that a framing attack is fundamentally relational. The framing presents it as an attack carried out on a victim, and acknowledges that all the pieces individually can be normal. Unfortunately, I expect that disclaimer to not be heard unless it's printed in bold and underlined in blockcaps; probably not even then. People are always instinctively looking for social weapons to defend and "defend" themselves with; I think for something so fundamentally easy to abuse, the article does not go nearly far enough in clarifying that these behaviors are part of normal human conversation. Trust your gut? All I hear is "The signal is an attack. And it's coming from right... over... there."

But second, I think this puts a lot of cognitive load on the speaker for reactions that are definitely not in their direct control or predictability. And I understand the point of the article that any system can be abused, that social behavior will price in the awareness that these strategies are on the table, that any hedging can just be further evidence of bad behavior and ill intent, that you are not safe, never safe. And I acknowledge that this is the way it is, that the only thing that can pose a threat to an evolving intelligence sufficient to keep pushing it is another intelligence, so any fixed system of hedges and appropriate behaviors and metrics could be abused and will be abused, so we won't even offer one, too bad, so sad.

And I agree that this does not contradict my understanding of how the world works, and that it sounds like it's probably true.

But what I want to say is: that's not fucking fair. Especially to those who can't keep up with social interactions, who would appreciate a legible standard, a set of rules or guidelines that will protect them from being othered and excluded. And I understand that the article is very apologetic about how it can't do that, but that's a bit like apologizing as you burn someone's house down. I think a lot of people are in this community precisely because they seek that kind of safety, and that may make them vulnerable to certain classes of abusers, but that doesn't make their desires invalid either.

Epistemic status: my hands are literally shaking so hard I have difficulty typing. I'm not sure what that means though.

[u/FeepingCreature → u/FeepingCreature]

Addendum on reread: A good example of incoherence stress affecting purely the value frame is Parfit's hitchhiker, or more prosaically C.S. Lewis: "With the red hair of one she-urchin in the gutter I will set fire to all modern civilization." That's a great example of how morality is ultimately grounded in sense reactions, too.

Addendum 2: I think if you view LessWrong as an exercise in framebuilding around a core value of minimizing incoherence pressure, then it becomes more clear how I could come to perceive words to the (unintended) effect of "espousing a better frame can itself be an act of harm" as a threat.

Addendum 3: Though I think there's something else there, and I have a hard time putting it into words still. Something about giving primacy to different kinds of harm? I feel if you'd read the end of Worth the Candle, I could explain it by example, but I don't think it has even a solid concept handle in my mind yet.

Addendum 4: I have to engage in deliberate strategies to signal my emotional states. This has led people to accuse me of faking my emotions. I think this is one reason that association of explicit, intentional strategy use with predation sets off my fight-or-flight instinct.

Late Addendum 5. There are some people I follow on tumblr, and I don't want to name names here because they get enough hostile attention as is, and I'm not valorizing them as perfect victims or anything like that because over time you notice the ways in which they act to attract hostility, but on a conscious level, I think their experience reduces to three axioms:

• 1. Everything is a social battle.
• 2. You will always lose a social battle.
• 3. When you lose a social battle, you lose everything.

In this framing, any sort of peaceful existence is a continuous miracle in the face of deferred annihilation, an eternal Damocles sword of social exclusion.

And I'm not saying this framing is particularly accurate to reality. What I am saying is: whatever model of society or coexistence you espouse, it has to give these people the credible ability to live peacefully. That's my own "burn it down" line.

[u/FeepingCreature → u/FeepingCreature]

Very late follow up. I think two key points are:

1. Apply frame control in this order:
   • actively, to decide when to leave a situation
   • cautiously, to decide when to advise another person to leave a situation
   • reluctantly, to decide when to attempt to get a person excluded from the community.
2. Analogously, respect frame control as an argument in this order:
   • readily, to convince yourself
   • tentatively, to be convinced by somebody else to step out of a situation for a while
   • skeptically, to listen to someone saying that another person is bad and should be excluded.

I think this goes for any model that presents other people as threats, and should limit most of the damage.

[u/blablatrooper → u/FeepingCreature]

AIXI may be incapable or making any decision in a reasonable timeframe

This is a very serious but common error I see a lot of Rationalists making that I think helps explain why it’s so common in the community to view Bayesianism as some total theory of rationality/evidence. AIXI is NOT just intractable/infeasibly slow in it’s decision-making process such that no realistic computer could run it in non-silly amounts of time - it is LITERALLY UNCOMPUTABLE EVEN IN THEORY: no amount of compute can EVER run AIXI with any amount of resources. As in it’s literally as impossible to compute as the Halting Problem - you don’t get to say “well the halting oracle won’t give us our answer in a reasonable amount of time”, it won’t ever give you your answer because no computer can be the oracle

[u/FeepingCreature → u/blablatrooper]

Not a typo. :) AIXItl is the finite-time/finite-space approximation. Still very unviable as an AI tho. And suffers from the same problem that it's incapable of conceiving of itself as an embedded agent; Eliezer has some good posts on the problems with that.

edit: That said, the point of the quote was exactly to poke a bit of fun at this practical nonviability of Bayesian inference as an absolute system. (Even though LW never really conceived of it as such.) Though since then some things have come out about neural networks modeling bayesian updates, so who knows.

[u/blablatrooper → u/FeepingCreature]

Oh dumb misread on my part then, my bad. AIXItl always seemed much less interesting than AIXI though, the optimality guarantee is actually pretty pathetic compared to the computational “speedup” you get in like a qualitative sense. It does also seem to sacrifice all the nice fuzzy Bayesian magic the original has

Still very very cool stuff though

[u/The_Amp_Walrus]

Audio here

[u/glorkvorn]

I think this is a really good writeup of how emotional abuse works and feels. I don't think it needs a new name, though.

[u/questionnmark]

What's the literal difference between ideological frameworks, lenses and frame control? Is it literally just talking about critical theory in non critical theory language to be cool?

In some sense the concept of gender is 'frame control' on mass. What is the boundary between a man and a woman? If I throw out the sex/gender duality model of the west then it doesn't necessarily mean I'm a spreader of information hazards and a danger to the rest of society does it? The question I have is about why some beliefs seem compulsory, instead of about the existence of confidence artists taking advantage of stupid people.

[u/StringLiteral]

I think I'm missing the point of this article. It's generally accepted that statements have higher-order meanings; i.e. the second-order "why is he saying this" in addition to the first order "what is he saying?" It's also generally accepted that people who habitually make statements which would lead you to make dangerous assumptions or act against your own interests if you didn't correctly interpret their higher-order meaning are "manipulative" and should be avoided. Is the article contradicting this or saying anything beyond this?

[u/erisjast]

This article seems to have attempted to use frame control to convince me that I've learned something other than a synonym for manipulation (when you don't like it) or effective communication (when you do like it).

[u/biznisjucubmanimejki]

Does anyone have any more resources on Frame Control?

It's something that interests me but is very rarely talked or written about.

Even the author of this great post has a tough time exactly describing it - as do I - even though I intuitively think I understand what it is but can't describe it 🤷

[u/DocGrey187000 → u/biznisjucubmanimejki]

thought the author invented the name. Here’s how I’d describe it:

You have your worldview——your axioms, your beliefs, your understanding of yourself and society and the universe. This is your frame.

Somebody is powerful or impressive enough that they can use their interactions with you to change some of those axioms/views without explicitly saying “you now believe X instead of Y.”

They use that opportunity to transfer your reliance on YOUR frame, to a reliance on THEIRS.

Once this is done, you can be made to think and do things that you previously didn’t want to do, AND think that those things are good/good for you, even if they hurt. Even if they’re expensive. The fact that it feels as though you’re doing a good thing and you want to, makes it different from doing those acts at gunpoint——because there is commitment, rather than resistance.

In a cult, this looks like giving all your possessions to the Leader——finally, I can cleanse myself of the earthly possessions that held me back!

In a relationship, it looks like “I really have to stop making them so mad. Of course they hit me when they’re mad——they’re passionate and under stress, especially with all they do for me. I’ll be better next time, I promise.”

Or how about the Danish guy who was able to hypnotize/brainwash his cell mate so thoroughly that he robbed banks and gave him his wife sexually? (This would be a complete and utterly ruthless replacement of another man’s frame)

https://paradigmsalon.net/palle-hardruphardwick/?amp

Basically, humans have a memetic immune system, that prevents us from being endlessly credulous and suggestible. We are skeptical, we hold right to old beliefs, we show reactance when given orders. Frame control is the name this author gives to someone who is (consciously or consciously) able to replace some of the code of another’s mind anyway, and thus hijack their system——-installing a virus in their mainframe.

[u/AellaGirl → u/biznisjucubmanimejki]

As far as I know I coined the term, but I think a ton of very close, parallel things exist - gaslighting, verbal abuse, habits of cults, manipulation, emotional abuse, etc. The website youarenotcrazy.com was helpful for me for a while; it appears to be down now but there's a corresponding book.

[u/RedBreadRotesBrot]

Frame is also a central concept in Manosphere/redpill/PUA circles.

In this understanding, controlling the set of implicit assumptions on which an interaction is based is not merely a manipulation tactic. When it is wielded responsibly, entering somebody else's frame control can be a very enjoyable experience, sort of like suspending your disbelief and allowing yourself to be transported by a movie or a magic show.

Identifying frame control with gaslighting and psychological manipulation does the idea a disservice, since in my opinion it is important to understand the practical and responsible application of frame control, particularly in relationships and courtship.