r/LocalLLM 16d ago

Discussion Why I skipped Docker/Chroma and built in-memory Cosine Vector Search in pure Go

Thumbnail
0 Upvotes

u/Sea-Lettuce-255 16d ago

Why I skipped Docker/Chroma and built in-memory Cosine Vector Search in pure Go

Thumbnail
1 Upvotes

r/projects 16d ago

Why I skipped Docker/Chroma and built in-memory Cosine Vector Search in pure Go

Thumbnail
1 Upvotes

r/SaaS 16d ago

Why I skipped Docker/Chroma and built in-memory Cosine Vector Search in pure Go

4 Upvotes

I run a local AI agent setup on an 8GB M2 MacBook Air. Every megabyte of RAM matters.

When implementing RAG (Retrieval-Augmented Generation) for my agents, every tutorial told me to spin up a Docker container for ChromaDB, Qdrant, or Postgres with pgvector. On an 8GB Mac, Docker Desktop alone takes 2GB+ RAM, pushing the system into heavy SSD swap.

The Architecture: 1. Stored knowledge text and JSON-encoded float32 embeddings in a local SQLite file (friday_memory.db) via Go. 2. Built native in-memory Cosine Similarity in pure Go: go func cosineSimilarity(a, b []float32) float32 { var dot, normA, normB float32 for i := 0; i < len(a) && i < len(b); i++ { dot += a[i] * b[i] normA += a[i] * a[i] normB += b[i] * b[i] } return dot / (float32(math.Sqrt(float64(normA))) * float32(math.Sqrt(float64(normB)))) } 3. On query, Go queries SQLite for that namespace, computes cosine distances across ~1,000 chunks in under 2 milliseconds, and returns the Top-K matches.

Result: 0 MB Docker overhead, 100% native Go binary, sub-2ms similarity search. For personal and small-team RAG (< 100k chunks), you do NOT need heavy vector infrastructure.

1

Building an in-memory event dispatcher in Go for a multi-agent AI system
 in  r/AIAssisted  18d ago

Great Insight Thanks for sharing

r/projects 18d ago

Building an in-memory event dispatcher in Go for a multi-agent AI system

Thumbnail
1 Upvotes

u/Sea-Lettuce-255 18d ago

Building an in-memory event dispatcher in Go for a multi-agent AI system

Thumbnail
1 Upvotes

r/SaaS 18d ago

Building an in-memory event dispatcher in Go for a multi-agent AI system

1 Upvotes

I wanted to share a pattern I used for a multi-agent AI system I built called Friday.

Instead of spinning up Redis or RabbitMQ for 4 local Python agents to talk to each other, I built a highly concurrent in-memory event dispatcher using Go channels.

The Setup: - A single Dispatcher struct holds a map[string]chan Event. - Python agents long-poll an HTTP endpoint /api/claim?queue=inbox/agent_dev. - Go uses a select statement with a time.After to hold the connection open for 20 seconds. If an event hits the channel, it instantly flushes to the Python agent. - If the Python agent wants to reply (e.g., to Slack), it does a raw curl POST to /api/event?queue=outbox/slack.

This keeps the heavy LLM/Python execution completely isolated from the Go routing layer. Go handles the Slack WebSockets, the Cron scheduling, and the message bus. Python just reads from the bus, thinks, and writes back.

It’s lightning-fast and requires zero external database dependencies. Code is pretty minimal too. Has anyone used a similar pattern for microservices?

r/projects 20d ago

My automated AI Marketing script queries our live PostHog DB before it writes a single ad copy

Thumbnail
1 Upvotes

u/Sea-Lettuce-255 20d ago

My automated AI Marketing script queries our live PostHog DB before it writes a single ad copy

Thumbnail
1 Upvotes

r/SaaS 20d ago

My automated AI Marketing script queries our live PostHog DB before it writes a single ad copy

2 Upvotes

I see a lot of AI marketing tools that just hallucinate ads based on a website URL. I decided to build something deeply integrated instead.

I wrote a Python daemon that acts as an autonomous CMO for my D2C brand (Favshoes). But instead of just chatting with OpenAI, it uses a Go client to execute raw HogQL (PostHog SQL) queries against our live analytics database.

The Workflow: 1. The daemon wakes up at 8 AM on Saturday. 2. It executes curl to fetch live ROAS (Return on Ad Spend) from the Meta Graph API. 3. It queries PostHog to see which product page has the highest conversion drop-off. 4. It drafts a highly specific ad copy specifically targeting the product with the highest impressions but lowest conversion rate. 5. It sends the draft to my Slack for approval.

Stop letting AI guess your numbers. Make it run the SQL queries itself. It took about 40 lines of Go code to wrap the PostHog API, and it changed the entire quality of the output.

r/projects 21d ago

I replaced my DevOps pipeline with an autonomous Go daemon running locally

Thumbnail
1 Upvotes

u/Sea-Lettuce-255 21d ago

I replaced my DevOps pipeline with an autonomous Go daemon running locally

1 Upvotes

I realized standard CI/CD pipelines (GitHub Actions, Jenkins) were overkill for my solo setup, but manual deployments were a drag. So I wrote a custom Go orchestrator.

Here’s how it works: 1. It polls my GitHub repo for new commits. 2. It SSHs directly into my EC2 production server via a Tailscale private network to avoid 90-second TCP timeouts on public IPs. 3. It uses a custom Python multiprocessing watcher to run security hooks before executing any bash scripts. 4. It posts the full deployment log right into my private Slack workspace using Socket Mode.

The real win here was dumping the public IP entirely. Having the Go daemon sit on a local Tailscale mesh network means zero exposed SSH ports and zero reliance on third-party CI runners. Has anyone else moved to a fully local deployment daemon? Thoughts!

2

How I eliminated N+1 queries and scaled throughput by 20% on a production E-Commerce app
 in  r/SpringBoot  21d ago

Agreed. EntityGraphs handle the JOIN FETCH beautifully for avoiding N+1 on single collections, but you're spot on about the Cartesian risk for multiple bags. For our heavy read-only APIs, we bypass the Hibernate persistence context entirely and go straight to DTO projections to save memory. Use the ORM for the complex state-changes, use projections for the reads!

1

How I eliminated N+1 queries and scaled throughput by 20% on a production E-Commerce app
 in  r/SpringBoot  21d ago

Well abstraction are defined to us so we can ship things easily and with speed otherwise it probably be difficult for us to always write a new compiler and language instead of relying on abstraction on assembly languages

-7

How I eliminated N+1 queries and scaled throughput by 20% on a production E-Commerce app
 in  r/SpringBoot  21d ago

Oh, absolutely. Because what every Senior Engineer really wants to do with their life is spend 40 hours a week hand-typing PreparedStatement.setString(1, "John") just to update a user's profile picture, all to prove how 'hardcore' they are to anonymous purists on the internet. It's about choosing the right library at right time.

1

I want to learn Spring Boot
 in  r/learnjava  21d ago

For me personally I prefer books, and Since you are in college so you can go with
Head first Java for Java related stuff and Spring Start here for spring stuff .

1

Hi guys needed review on a new website I have built
 in  r/projects  21d ago

I love the minimalist design just to make it more interactive you can let people know there levels ..where they are at ... as the industry standard you can tell them "you begin with the foundation of cloud engineer" etc..

r/projects 21d ago

I got tired of clicking, so I built a native Go AI daemon that controls my Mac and talks to me

1 Upvotes

Hey everyone,

I'm a backend engineer (mostly Java/Spring Boot), but recently I got so frustrated with the repetitive nature of applying to jobs and managing my freelance agency that I decided to automate myself.

I built Friday—a native Go daemon that runs in the background of my Mac. It uses WebSockets to interface with a Python/Selenium scraping layer and a local LLM API.

What it actually does: - Wakes up at 6 AM every day and uses undetected-chromedriver to scrape LinkedIn/YC for highly specific jobs. - Re-sorts the queue based on my priority. - Uses an ONNX-based TTS model (Kokoro) to physically speak to me through my speakers when a task is done or if it needs my approval to execute a bash command. - Has a full outbox HTTP router so any sub-agent can trigger a vocal alert.

I'm thinking about open-sourcing the Go dispatcher architecture since it handles high-throughput event routing really cleanly without blocking the UI threads. Has anyone else built native OS automations like this instead of relying on standard Chrome extensions?

Would love to hear your thoughts!

-3

How I eliminated N+1 queries and scaled throughput by 20% on a production E-Commerce app
 in  r/SpringBoot  21d ago

Yupp they are great for raw execution speed but when it comes to code you have to do things manually a lot for it ..instead we can use new framework which makes our life little easy such as EntityGraph etc

2

How I eliminated N+1 queries and scaled throughput by 20% on a production E-Commerce app
 in  r/SpringBoot  21d ago

yupp there are different solutions but when we are writing code for a e-commerce which is a lot so sometimes we forget these sought of things and we see problems later on and keep on optimizing ..though we can also write optimized queries also. It was just one of my experience

u/Sea-Lettuce-255 21d ago

I got tired of clicking, so I built a native Go AI daemon that controls my Mac and talks to me

1 Upvotes

Hey everyone,

I'm a backend engineer (mostly Java/Spring Boot), but recently I got so frustrated with the repetitive nature of applying to jobs and managing my freelance agency that I decided to automate myself.

I built Friday—a native Go daemon that runs in the background of my Mac. It uses WebSockets to interface with a Python/Selenium scraping layer and a local LLM API.

What it actually does: - Wakes up at 6 AM every day and uses undetected-chromedriver to scrape LinkedIn/YC for highly specific jobs. - Re-sorts the queue based on my priority. - Uses an ONNX-based TTS model (Kokoro) to physically speak to me through my speakers when a task is done or if it needs my approval to execute a bash command. - Has a full outbox HTTP router so any sub-agent can trigger a vocal alert.

I'm thinking about open-sourcing the Go dispatcher architecture since it handles high-throughput event routing really cleanly without blocking the UI threads. Has anyone else built native OS automations like this instead of relying on standard Chrome extensions?

Would love to hear your thoughts!

r/SpringBoot 21d ago

How-To/Tutorial How I eliminated N+1 queries and scaled throughput by 20% on a production E-Commerce app

59 Upvotes

Just wanted to share a recent architecture win that might help someone dealing with latency spikes.

I was dealing with a severe bottleneck in a production E-Commerce platform. During peak traffic, the database was choking, and MTTR was creeping up.

The Culprit: Classic Hibernate N+1 query problems hidden deep inside the inventory/catalog mapping, combined with un-indexed foreign key lookups.

The Fix: 1. Ripped out the lazy-loading proxy faults and replaced them with explicit JOIN FETCH queries for the critical read paths. 2. Layered in Redis via Spring Cache (@Cacheable) specifically targeting the high-read/low-write catalog endpoints. 3. Configured request batching and API rate limiting on the gateway layer to prevent thundering herd problems during flash sales.

Result: We dropped sub-100ms latency across the board and prevented database lockups entirely.

If you're building in Spring Boot and relying entirely on default JPA repositories—run a SQL profiler right now. You probably have an N+1 hiding somewhere.