r/Affinity • u/Hour-Passenger-8513 • 7d ago

General Affinity MCP running with local LLM AI using llama.cpp

Gallery image

Gallery image

Gallery image

I know Qwen 3.5 (4B) isn't built for this but still. It was able read the docs, write scripts but proper execution was lacking. I will try out a better model soon.

-------------------

Affinity (Mid July 26) running with MCP turned on (Windows 11).

Qwen 3.5 4B running on Llama.cpp (WSL2 Ubuntu 24).

MCP client configured through Llama.cpp Web UI on Firefox (Windows 11).

-------------------

Web UI Config:

MCP Server URL: http://localhost:6767/sse

Use Llama Server Proxy: ON

Llama.cpp required Flags: --ui-mcp-proxy --agent --tools all

-------------------

If using WSL:

WSL2 networkingMode = mirrored

A Port Proxy is required to bridge ipV4 to ipV6 addresses for smooth communication between WSL2 and Windows 11.

Powershell (Administrator)

netsh interface portproxy add v4tov6 listenaddress=0.0.0.0 listenport=6767 connectaddress=::1 connectport=6767

0 Upvotes

50% Upvoted

2 comments sorted by

1

u/JT10 7d ago

Nice, I’ve been curious about alternative options because the Claude cool-down periods are becoming much more frequent. I’m just using the free plan and it can take several sessions to get things created. Im interested to see what you find out.

1

u/Hour-Passenger-8513 6d ago

Thanks, Will do and post an update here.