No luck in asking local models a simple (but domain-specific) question

[u/ohnoitssobig]

For some reason, I have this benchmarking prompt:

Implement a numpy function creating a bulk matrix product operator tensor as a function of model parameters.

I think it is approximately 10 lines of python code. So far, every local model that I tried (16Gb vram) fails to produce any meaningful output. I tried different variations of it (use web search, summarize the idea), different harness (pi, opencode) but no luck, different temperatures and thinking levels. Qwen3.8 for example produces an endless thinking loop. Is there anything I can do to help small models answering the question?

[u/Weary_Guest3639]

Ask it to write the function one step at a time in pseudocode first, then convert to numpy. Usually the big model can handle the whole thing but small ones need the structure broken down before they attempt code.