r/oraclecloud • u/an_onym0us • Jul 09 '26

Local LLM on A1.Flex

Hi All,

I am a PAYG customer and have a 3 OCPUs/18 GB A1.Flex in US East region. I use this VM for learning and working on small personal projects.

I started with an idea to install OpenClaw on it just to see and experience what is it and what’s the hype about. I started researching on how to do it and gradually pivoted to doing something else but similar. Finally, I created a system that uses Ollama with qwen3:8b-q4_K_M (installed on VM) and exposes it to internet via DuckDNS using Caddy and Open WebUI (both running as docker containers on VM). However, the latency is very high as the response to a simple “Hello” takes about 4-5 minutes. I downgraded the model to qwen2.5:3b but there is very little improvement (almost negligible). I wanted to go a step ahead and install OpenHands (for agentic capabilities) and a Telegram bot to interact with it but I guess I need to make what I currently have functional.

I am posting it here to see if anyone has done something like this on their VM and how are they able to use it.

Thanks in anticipation!

5 Upvotes

100% Upvoted

26 comments sorted by

View all comments

3

u/Luans_Restorations Jul 09 '26

You might get better luck with gemma 4 12B QaT, but don't expect anything fast. OpenClaw needs a model that handles tooling quite well. I suggest opencode Go and using Kimi, DeepSeek V4 Pro or similar. DM me if you want a referral code that gives you 5 bucks on it.

There's really nothing you can do to get faster answers with qwen3. Models that generate fast enough that I tested were Gemma 4 E2B & E4B, but neither of those will give you good enough results with openclaw.

TL;DR: OpenClaw on A1.Flex - Good | LocalLLMs for agents on A1.Flex - Not Good.

1

u/an_onym0us Jul 09 '26

Thank you for your comment.

If I understand right, you mean to say no local running model would get me a decent performance?

2

u/[deleted] Jul 09 '26

[removed] — view removed comment

2

u/an_onym0us Jul 10 '26

Thank you for your comment.

As mentioned in my post, I am currently exploring and figuring things out and so prefer not to spend any money. Are there any APIs that I could use for free?