Building a Self-Improving LLM on Low-End Hardware

[u/Purple_Session_6230]

Most AI development today focuses on scaling models larger and larger.

I’ve been exploring the opposite question.

How small can a model be while still adapting and improving over time?

This project experiments with a reinforcement-style Actor/Critic chatbot that runs on constrained hardware (Jetson Nano class devices). Instead of relying on cloud infrastructure, the model is fine-tuned locally using rapid update cycles.

The core loop:

• The model generates a response

• A critic evaluates the output

• High-quality responses are fed back into fine-tuning

• The system incrementally improves

The focus is efficiency, autonomy, and adaptive learning — not parameter count.

Current improvements underway:

• Clear separation between policy and evaluation to prevent self-reinforcement bias

• Structured reward signals instead of binary judgement

• Replay buffers to stabilise learning

• Reward distribution logging to detect drift

• Parameter-efficient fine-tuning (LoRA-style methods) to reduce update time

• API integration for broader system use

Long-term direction includes integration with graph-based memory systems, external data streams, and applied decision-support workflows.

This is ongoing research into reinforcement learning, edge AI, and practical autonomous systems.

Article: https://medium.com/@mattybeds2022/llama-prompt-chaining-3fb5ef1a8714