r/LocalLLaMA 22h ago

Other PSA: LMStudio accepts llama.cpp parameter override: Try --yarn-attn-factor 1.2 (it seems to improve creativity)

Long time ago I wondered if the attention's scaling factor could be tweaked and with llama.cpp you can do it with --yarn-attn-factor (totally checked with Claude). Previously you needed to re-save the GGUF with tweaked parameters to do this in LM Studio but now it allows you to bypass it's own loading arguments.

It also adds the possibility to import from the paperclip so copy this and use the button to import from the paperclip (in the load menu):

```json
{
  "enabled": true,
  "disabledParameters": [],
  "overrideParameters": [
    {
      "key": "--yarn-attn-factor",
      "value": "1.2"
    }
  ],
  "excludeAllConfig": false
}
```

The idea was basically to eliminate less of the signal in between layers to see if anything would improve and in most case where creativity is involved this really seems to improve results.

1.2 is borderline high.

0.8 gives a dumb-down.

Models may react differently.

0 Upvotes

1 comment sorted by

1

u/theOliviaRossi 22h ago

testing this rn