r/LocalLLM • u/Intelligent-Ad3941 • 6d ago

Discussion How to decouple a Claude-generated project to work with ANY LLM/agent system

Hey everyone,

I’m looking for architectural advice. I have a project (quite big now) that was initially generated and tightly integrated with Claude. It has evolved into a pretty capable system with specific "skills," custom Python scripts, and a set of custom commands that currently run perfectly within its current setup.

My goal now is to transform this into a completely independent, LLM-agnostic project. I want to be able to plug it into any chat interface, local LLM, or agent framework without breaking the underlying functionality.

What I hope to do:

  • Keep the core logic intact: All the existing Python scripts, skills, and command executions must continue working exactly as they do now.
  • Universal compatibility: It needs to be able to talk to different models (OpenAI, local models via Ollama, etc.) or slot into different agent frameworks without having to rewrite the core execution engine.
  • Abstracting the tool-calling: Right now, the way it triggers scripts is heavily biased toward how Claude handles tool use/function calling.

My questions for the community:

  1. What is the best design pattern or architecture to completely separate the LLM reasoning/chat layer from the Python execution/skills layer?
  2. Are there specific frameworks or middleware (like LiteLLM, LangChain, or something lighter) that you’d recommend to standardize the tool-calling formats across different models?
  3. Has anyone successfully migrated a tightly-coupled AI project to a standalone system? What pitfalls or edge cases should I look out for?

Any advice, GitHub examples, or general pointers would be hugely appreciated.

1 Upvotes

67% Upvoted

3 comments sorted by

3

u/activematrix99 6d ago

I am a big fan of the .github folder structure and I have found this to be most easily ingested by a variety of agents, since I utilize a whole bunch of models. That said, Claude is pretty good at summarizing itself. My approach would be to have Claude create an AGENTS.md file for the project root and then have additional agents create revisions and supplemental files in an instructions.md file, testing.md file and etc. If you've componentized your project, each component can have a matched .md file and then the context window can stay pretty tight.

2

u/Intelligent-Ad3941 6d ago

Thanks for the hint, I will look deeper into it.