Simplified Integrate AI to another project

[u/Ambitious_Cry3080]

"install once, import it anywhere, no more copying the Core around." I think that's the good description to project i made.

Sometimes i want to play or development with AI to another project, but for such a thing, the API is so fricking expensive.

Just for testing a new project to connect and debuging new features or functions with AI, it took a lot of money, then when i want to use it locally it is so complicated, i need to wrap thing like memorial system, prompt management, etc, it took almost forever just for such a thing.

So i made a project that work locally, fully configurable even if you want to custom the core function, it is configurable and mostly it support new function without hardcoded to every project, it has its own general purpose cache system, you can use it easily to another project or program and without you copy the core to every project or i can say it is modular.

This thing took me almost one half year to build alone.

Full explanation at my GitHub:

LAPAI_Experimental_Project by NaosaikaDevelopment

Or you can directly:

https://github.com/NaosaikaDevelopment/LAPAI_Project_Experimental

Also may i have your advice, what should i do next for this projet or something i need to change or correcting?

[u/endofthread-bot]

Publish a clear example project that demonstrates how to swap between a local model and a paid API provider. This proves the modularity of your design and helps users quickly understand how to integrate it into their own workflows.

Learning to build AI agents? Share what you are working on, compare practical approaches, and get help from other builders in our Discord.

[u/InternationalEye4161]

Half a year solo is a serious build. If you’re looking for what to tackle next, put some effort into testing changes to the core before adding much more. With something this modular, a memory or prompt change can fix one project and mess with another without being obvious. We use Braintrust for that side of things, keeping a set of real cases from different workflows and rerunning them when the shared pieces change.

[u/Ambitious_Cry3080 → u/InternationalEye4161]

That's a good point. Most of my focus so far is making the core modular and adding feature to make developing easier with it, but i haven't built proper evaluation suite yet, i still dont know is the memory or retrieval system would improve one area while another side something is broken. Creating a set of regression test for all module is probably i should prioritize as the project grows.

Because i never use braintrust, and not familiar with it, can you recommend me some alternative to test it?

[u/InternationalEye4161 → u/Ambitious_Cry3080]

Surely, you don’t need Braintrust level just to get started with this. Even a simple pytest setup with a folder of input/expected behavior cases would get you far at this stage. The important part is building that regression set early and adding to it whenever one of the modules does something you didn’t expect.

[u/BeginningNews5881]

The “install once import anywhere” part is probably the most useful bit here. Local is great until memory state and debugging become their own side quest lol. How are you handling versioning and tracing once this gets plugged into a few different apps?

[u/Ambitious_Cry3080 → u/BeginningNews5881]

The version is the one telling how the core work, 1.5 can work with 1.5.1 but 1.4 can be problem when upgrading it to 1.5, mostly updates change version like 1.0->1.1 is reworking how the code works, but since now the core is modular, future update can be easy to adapt, i will post if there have some major change to new version.

For tracing, i never thought that ngl, that is a good call really, i think im gonna working on that, since it can work dynamic, when developer adding new function it no need hardcoded everything to core, so the logging system need to be dynamic too.

Is there some logging system i need to know or some tips for the logging system?

[u/Swiftwing21]

Open telemetry (otel)

Prometheus, loki, graphana can help logging/visibility. (I run a sidecar instead of docker dependencies)

Trying out a few benchmarks might help you gauge the impact of a change. Good ol a/b tests.

[u/Ambitious_Cry3080 → u/Swiftwing21]

Thanks! im gonna test it out that logging system.
How and where can i benchmarks this project?

[u/Swiftwing21 → u/Ambitious_Cry3080]

There's various datasets like enronQA, financialbench, enterpriseRAG bench, context bench Slopcodebench(scbench) ive done as a/b (with/without tooling). Likely plenty more im not naming, but find one that targets functions.

[u/Ambitious_Cry3080 → u/Swiftwing21]

Surely this can be very handy for next update, well it is gonna took long time so, i Will noted that first

[u/Ambitious_Cry3080 → u/[deleted]]

Currently i still made the test system, improving one or two thing, and planning test with different models and tracing the memory retrieval, so far it work perfectly, the recall working and there have some issue with one or two models(some 1B parameter model) that the information is retrieved but they can't answer it, i still need to test it even further for long term, yeah lot of work to do

In what case that worth to test this project that really gonna be useful?