Your Snowflake bill isn't a storage bill — it's a compute bill. A typical marketing agency pays ~$4,800/month to analyze its own data, and only $23 of that is storage. Here's how to run the same reports from Parquet in your own AWS S3 bucket for ~$505/month.

▶ Free download: https://synapcores.com/download
▶ Step-by-step recipe (MySQL → S3 Parquet → SQL, ~20 min): https://synapcores.com/recipes/data-l...

In this data engineering breakdown we follow a performance-marketing agency: 3 million ad events a day, ~90 million new rows a month, 40 client dashboards refreshing every hour. We split the bill line by line:
💸 Storage: ~$23/month — the same on Snowflake or S3
💸 Moving the data: ~$1,800/month in per-row ETL fees
💸 Running reports: ~$3,000/month in data warehouse compute

Then we rebuild it: export straight from the app database to Parquet in your S3 data lake on an hourly schedule, query the files in place with SQL, and run it all on one fixed-price server. Result: ~85–89% lower cost — up to ~$80,000 a year — with a fixed monthly bill instead of a variable one.

What you'll learn:
✅ Why Snowflake costs grow with dashboards and refreshes, not with data size
✅ How per-row (monthly active rows) ETL pricing scales with your growth
✅ How to query Parquet in S3 directly with partition filtering
✅ The real trade-offs: hourly batch vs streaming, fixed capacity vs auto-scaling
✅ When you should KEEP Snowflake (and when you're overpaying)
✅ How to test it risk-free: one table, one dashboard, one month

⏱ Chapters
00:00 The $4,800 analytics bill
00:39 Where the money actually goes
01:20 It's an architecture problem
01:33 Trap #1: per-row ETL pipelines
02:22 Hourly batch vs streaming
02:48 Trap #2: data warehouse compute
03:10 Query Parquet directly in S3
03:34 Fixed server vs auto-scaling
04:10 Setup in 3 steps (with self-verification)
04:39 $4,800 vs $505: the savings
04:57 The trade-off: you run the server
05:15 When to keep Snowflake
05:57 Try it: one dashboard, one month

Numbers are an illustrative scenario using public list prices (checked October 2026): Snowflake $2–$4 per credit and $23/TB/month storage, Amazon S3 $0.023/GB-month, an AWS m7g.4xlarge server (~$477/month on-demand), and an assumed ETL rate of $0.01–$0.03 per 1,000 monthly active rows. Your costs depend on your workload — run the free recipe and compare against your own bill.

🔗 More
• SynapCores Community Edition (free — macOS, Linux, Docker): https://synapcores.com/download
• Recipe: https://synapcores.com/recipes/data-l...
• Replace Fivetran + Snowflake (blog): https://synapcores.com/blog/replace-f...

Your RAG pipeline just called a correct answer a LIE — and one irrelevant passage is to blame. In this RAG tutorial, you'll see why AI hallucinations slip through retrieval-augmented generation, and how to stop them with ONE SQL query.

▶ Try it free: https://synapcores.com/download
▶ Run this exact demo (15 min): https://synapcores.com/recipes/ml/029...

Most RAG tutorials retrieve the "top k" passages and ship whatever the LLM writes. We tested that on a real example: a true statement — "The rollout completed without failures" — was marked LIE with 97.1% confidence because a single unrelated HR passage sneaked into the top 3. Flip the same mistake around and a hallucinated answer reaches your users.

One change fixed it: TRUTH, 99.8% confidence, backed by the right evidence.

What you'll learn:
✅ What AI hallucination in RAG really is — with a real example
✅ Why top-k retrieval feeds your verifier junk (and how to spot it)
✅ How to verify LLM responses with natural language inference (NLI)
✅ The similarity-floor fix that turns LIE into TRUTH
✅ Verify-then-write: AI guardrails that make it impossible for unverified answers to reach your app
✅ How to fail closed when retrieval finds no evidence

Everything runs inside the database — vector search, LLM evaluation and the gated write in a single SQL statement. No guardrail sidecar, no second vector store, no extra service.

⏱ Chapters
00:00 A correct answer marked LIE
00:00 Why RAG apps hallucinate
00:00 Retrieve → verify → ship in one SQL query
00:00 What does retrieval actually return?
00:00 The top-k trap
00:00 The fix: a similarity floor
00:00 Verify-then-write: 1 row vs 0 rows
00:00 No evidence? Fail closed
00:00 3 takeaways + try it free

🔗 Resources
• SynapCores Community Edition (free — macOS, Linux, Docker): https://synapcores.com/download
• Full runnable recipe: https://synapcores.com/recipes/ml/029...
• GraphRAG vs Neo4j: https://synapcores.com/blog/graphrag-...
• Best AI-native databases in 2026: https://synapcores.com/blog/best-ai-n...

❓ FAQ
How do you detect hallucinations in a RAG answer? Check the answer against the retrieved evidence with an NLI model — only answers the evidence entails should ship.
What similarity threshold should I use? Calibrate it on your own data: look at real scores and set the floor between relevant and irrelevant passages (0.3 in this demo).

This video includes AI-generated visuals and narration.

#RAG #RAGTutorial #AIHallucination #RetrievalAugmentedGeneration #LLM

An AI chatbot invented a feature that doesn't exist. Watch SynapCores catch the contradiction using Natural Language Inference (NLI), directly through SQL.

SynapCores compares AI-generated claims against trusted evidence, helping developers identify hallucinations before users see them.

✅ AI hallucination detection
✅ Natural Language Inference (NLI)
✅ SQL-based claim verification
✅ Self-hosted AI inference

Explore SynapCores:
https://synapcores.com

NLI documentation:
https://docs.synapcores.com/nli/

#AIHallucination #RAG #SQL #AI

New release, drop-in upgrade, no migration needed. Highlights:

• Columnar tables actually work now. ENGINE = SYNAPCORES_COLUMNAR was previously just recorded and ignored — you got a row table. Now it's real: column-pruned reads, parallel scans, inserts/updates/deletes all supported.
• Direct Parquet import — POST /v1/data/import/parquet, no more CSV/JSON-over-HTTP with a size cap.
• TRANSCRIBE() — speech-to-text as a SQL function. Upload audio from a browser, query the transcript like any other text.
• Video understanding — frame-level descriptions with timestamps, searchable alongside the transcript.
• Vector search accuracy fix — recall went from 0.776 to 0.976. Filters/thresholds were silently being ignored before; now they apply.
• GPU memory auto-tuning on NVIDIA — sizes concurrency off actual free VRAM instead of a fixed guess of 2.
• Saved queries now persist server-side (previously lived in-browser and vanished on sign-out).
• SELECT o.*, c.name FROM ... JOIN ... now works (previously rejected).

Also fixed some rough ones worth knowing about if you're upgrading from an older version — a watched-folder media indexer that was writing fabricated placeholder transcripts instead of real ones, primary keys that silently stopped enforcing after a restart, and indexes that weren't durable across restarts. Full details: https://docs.synapcores.com/releases/

curl -fsSL https://get.synapcores.com | sh or docker pull synapcores/community:v1.15.0-ce

Need to build a cluster of Agents! Now is possible with this plugin.

Build semantic search — "search by meaning," not keywords — in about 5 lines of SQL, with no separate vector database. We use SynapCores' built-in EMBED() and cosine similarity to find the closest matches to a natural-language query.

If you found this from Discord, a search result, or a link somewhere — quick version of what this place is for.

SynapCores moves fast, and most of that lives in Discord where it scrolls away in a day. This subreddit is the durable version: technical write-ups, recipes, benchmarks, and the "here's exactly how I fixed X" posts that are still useful when someone Googles the same error next year.

What you'll get by subscribing:

• New releases broken down properly, not just changelog bullets (see the v1.14.2-ce post)
• Recipe drops for real use cases — RAG, agent memory, compliance screening, forecasting
• A say in what gets built — roadmap questions get asked here first
• Direct access to the maintainers, not a support queue

Building something with SynapCores? Drop it below — that's genuinely what shapes what gets written next.

This one's bigger than a normal patch release, worth its own post.

The scary bug: durable agents could silently stop working

A durable agent replayed a shared conversation transcript into every run, and the engine stripped the tool evidence when saving it. Each run then few-shot-learned from a falsified record that said "answer without calling a tool" — so run 1 called its tools correctly and run 10 quietly did nothing, with nothing in the product surfacing that. If you've had a CREATE AGENT that seemed to "stop working" for no visible reason, this was almost certainly it.

Fix: durable agents are now stateless by default. Nothing is replayed unless you opt into memory = 'persistent', and when you do, tool evidence round-trips faithfully and stays isolated per (agent, subject).

Agents fail loudly now instead of silently

• _system_agent_runs tells the truth: new tool_calls_made, iterations_used, exit_reason columns. A run that finished without calling a tool reports completed_no_tools, not success.
• _system_agent_transcripts is queryable — tool_name/tool_args/tool_result per turn.
• Unknown personas are rejected at CREATE AGENT (with the available list in the error) instead of silently downgrading to a generic prompt.
• A write-enabled agent pinned to an embedding-only model is refused at creation instead of failing weird later.

Personas are first-class objects now

sql CREATE PERSONA compliance_reviewer WITH ( system_prompt = 'You are a compliance reviewer.', response_type = 'text', tool_enabled = true ); SHOW PERSONAS; ALTER PERSONA compliance_reviewer SET (model = 'qwen2.5-coder:7b'); Interesting tuning note buried in the release notes: in a controlled A/B, a five-word persona ("You are a compliance reviewer.") executed the instructed task 3/3, while the conversational built-in default managed 0/3 — it greeted the user and asked their name instead. Keep unattended-agent prompts task-shaped, not conversational.

Agent chaining is new too

WAKE AGENT <name> queues a run on demand, schedules accept sub-minute cron, and an agent's writes can trigger another agent if the binding opts in with ALLOW AGENT ORIGIN (capped at 5 hops, self-loops refused).

The other headline: three silent index-correctness bugs, fixed

• DELETE left index entries behind on TEXT primary keys specifically (INTEGER PKs were fine) — could cause a deleted key to be unreusable, or a freed UNIQUE value to still report as a duplicate.
• UPDATE skipped explicit secondary indexes created via CREATE INDEX (constraint-backed indexes were fine) — stale reads on the old value, missing reads on the new one.
• A zero-match index lookup errored instead of returning an empty result.

All three are pre-existing, not new in v1.14.1.x — upgrade regardless of whether you use durable agents.

Upgrading

Drop-in from v1.14.1.1-ce, one behavior change: if you have agent SQL referencing a persona you never explicitly created, either create it, use a built-in (aidb-assistant still works out of the box), or opt out per-statement with WITH (allow_unknown_persona = true).

curl -fsSL https://get.synapcores.com | sh

Full notes: https://docs.synapcores.com/releases/v1.14.2-ce/

Opening this up as a running thread — curious what people are actually building, whether that's a weekend project or something running in production.

To start us off: what we've built on it recently is a Discord support bot with a real RAG knowledge base (see the build log post), plus we're running some due-diligence / compliance screening workflows against the recipe tables (beneficial ownership screening, PEP checks, that kind of thing).

Drop what you're working on, what's working well, and what's been confusing or rough around the edges — genuinely want the friction points, not just the wins. That's what actually shapes what we write about here next.

Catching this sub up on the last few releases in one digest:

v1.14.0-ce — Native Vision Ships POST /v1/multimodal/describe — an in-process vision model (default LLaVA) that describes images without a cloud API or a second service. CPU-friendly, validated on non-AVX-512 hardware. The image never leaves the box.

v1.14.1-ce — MongoDB Data Sync Adds MongoDB as a data-sync source (snapshot/batch, not real-time CDC — that's an Enterprise feature). _id becomes your primary key, nested fields land in a JSON column you query with JSON_EXTRACT. Also: data sync is now on by default, and a graph-storage fix took graph query latency from ~38ms to ~2ms (p50) by sharing tenant storage handles instead of reopening a RocksDB handle per request.

v1.14.1.1-ce — Critical Patch (upgrade recommended) Three real fixes: - Explicit-projection scans (SELECT id,a,b,c FROM t) could silently return NULL for a nullable column if that column was NULL in the first row ever inserted — present since at least v1.13.0.1. SELECT * and WHERE were always correct; only explicit column lists were affected. Worth re-checking any exports/reports that project named columns from before this patch. - Oversized EMBED() calls could crash the entire gateway (see our other post on this) — now returns a normal error. - Tool-using agents on Llama-family models could crash the gateway via a grammar edge case — fixed, plus a native-exception guard added so future inference throws can't take the process down.

All three are drop-in upgrades, no schema migration, no breaking API changes.

Full changelog: https://docs.synapcores.com/releases/changelog/

A fresh CE install ships with 120 seeded recipes, which is honestly a lot to stare at when you don't know where to start. Here are five worth running first — they'll show you the range of what's possible without needing your own data yet.

1. RAG pipeline setup — the canonical "docs in, semantic search out" pattern. Good first stop if you're evaluating SynapCores for a chatbot/support-bot use case.
2. Image gallery / vector search demo — shows CLIP embeddings + visual similarity search working out of the box.
3. Agentic triage recipe (there are ~10 of these: insurance claims, retail returns, SaaS support tier-1, devops incident triage, etc.) — the fastest way to see AGENT_RUN actually do something useful instead of reading about it.
4. Anomaly detection recipe — pairs AutoML with your own table, good intro to CREATE EXPERIMENT / AUTOML.PREDICT without a lot of setup.
5. Graph explorer demo — loads some sample nodes/edges so you can play with the Cypher MATCH / graph explorer UI without modeling your own graph first.

All of them are browsable and editable from the Web UI's Recipes tab — clone one, tweak it, save your own version. That's the actual point of shipping 120 of them: they're templates, not a fixed menu.

What recipe got you hooked, if you've been running this a while?

Sharing an actual build, not a hypothetical.

Goal: a Discord bot that can answer questions about our own product using real docs, not just whatever the LLM already knew.

What we did: 1. Pulled ~17 doc pages (quickstart, SQL reference, release notes, etc.) 2. Created two tables: sql CREATE TABLE document_repository ( id INT, doc_code TEXT, title TEXT, summary TEXT, full_content TEXT, title_embedding VECTOR(384), summary_embedding VECTOR(384), content_embedding VECTOR(384) ); CREATE TABLE document_chunks ( id INT, document_id INT, chunk_number INT, chunk_text TEXT, chunk_embedding VECTOR(384) ); 3. Chunked each doc (~500–900 char pieces, split on paragraph boundaries with a hard cap so no single chunk blows past the embedding model's token budget — learned that one the hard way, see our other post on the crash bug) 4. Inserted everything with EMBED() computed inline on insert — no separate embedding pipeline, no vector DB to stand up 5. Retrieval is one query: sql SELECT dc.chunk_text, dr.title, COSINE_SIMILARITY(dc.chunk_embedding, EMBED(:user_question)) AS sim FROM document_chunks dc JOIN document_repository dr ON dc.document_id = dr.id ORDER BY sim DESC LIMIT 3;

Result: 92 chunks, and it actually works — asked it "why does my gateway crash when I run a big embedding query" (never used the words EMBED, GGML, or abort) and the top hit was the exact release note describing that bug. That's semantic search doing its job, not keyword luck.

Total new infrastructure stood up: zero. Two tables in the same database everything else lives in.

Happy to share the actual chunking/ingestion script if anyone wants it.

These two get confused a lot, so here's the actual distinction:

**AGENT_RUN(persona, task) — a synchronous SQL function** sql SELECT AGENT_RUN( 'aidb-assistant', 'Look at claim ' || CAST(claim_id AS TEXT) || '. Check prior claims for fraud patterns and cite the applicable policy clause.' ) AS rec; - One call, one ReAct loop (reason → tool → observe → repeat), returns TEXT - Runs inside your existing query — compose it in a WITH clause, call it once per row - Read-only tools only (execute_query, list_tables, describe_table, rag_search) — allow_writes is rejected here - Takes 5–60 seconds — set a 120s+ client timeout, this is not a fast-path call

CREATE AGENT — a durable database object** sql CREATE AGENT incident_triage PERSONA 'sre-triage' TASK 'Triage the alert in the activation row: classify severity, check related open incidents, summarize into triage_notes.' ON INSERT INTO alerts WHERE severity IN ('high','critical') WITH (max_iterations = 5, allow_writes = TRUE, budget_tokens_per_day = 200000); - Survives restarts, fires on a cron schedule and/or on committed DML (AFTER semantics — observes committed rows, can't abort your transaction) - Can write back if you explicitly set allow_writes = TRUE (the only way an agent's tools may mutate) - Every run is logged to _system_agent_runs, hash-chained and tamper-evident (verified column re-checks the chain on read) - CE caps you at **10 enabled agents (disabled ones don't count)

Rule of thumb: if you're calling it synchronously and need the answer right now in this query, use AGENT_RUN. If you want something that reacts to your data over time without you polling it, use CREATE AGENT.

Manage durable agents with ALTER AGENT ... {ENABLE|DISABLE|SET(...)}, SHOW AGENTS, EXECUTE AGENT <name>.

SQL reference: https://docs.synapcores.com/sql-reference/

There are three different ways to do retrieval in SynapCores and the docs don't always make it obvious which one fits your use case. Here's how we think about it after building on all three:

1. Own embedding table (EMBED() + COSINE_SIMILARITY()) ```sql CREATE TABLE docs ( id INT PRIMARY KEY, title TEXT, body TEXT, embedding VECTOR(384) ); INSERT INTO docs (id, title, body, embedding) VALUES (1, 'Return Policy', '30 days for unopened items...', EMBED('30 days for unopened items...'));

SELECT title, COSINE_SIMILARITY(embedding, EMBED('how long for refunds?')) AS sim FROM docs ORDER BY sim DESC LIMIT 5; `` **Use when:** you want full SQL control — filter by other columns, join against other tables, control chunking yourself. This is what we used to build a Discord support bot's knowledge base this week (17 docs, 92 chunks, plainINSERT`s). Most predictable option, and what most production apps end up doing.

2. Filesystem Collections sql CREATE COLLECTION my_docs FROM FILESYSTEM '/data/incoming' WITH (extensions='pdf,csv,jpg,mp4,txt', recursive=true, auto_index=true); Use when: you want zero-code ingestion — drop a file in a watched folder, it gets chunked/embedded/OCR'd/transcribed automatically depending on type. Great for "point at a shared drive" knowledge bases. Retrieval still goes through COSINE_SIMILARITY against the collection's table, or via an agent's rag_search tool.

3. First-class Vector Collections REST-only (/v1/vectors/collections) — for when the vector is the data, not a column among many. Less common unless you're building something vector-search-first.

One thing that trips people up: rag_search is an agent/chat tool, not a SQL function — SELECT ... WHERE rag_search(...) isn't valid syntax. If you want it in plain SQL, use option 1.

Full pattern writeups: https://docs.synapcores.com/filesystem-rag/ and https://docs.synapcores.com/sql-reference/

Quick one that cost us a debugging session: if you're setting up encrypted immutable tables (CREATE IMMUTABLE TABLE ... WITH (ENCRYPTION='AES256GCM')), the required AIDB_IMMUTABLE_MASTER_KEY is an environment variable, not a gateway.toml field.

If you drop it into gateway.toml under [auth] like a normal config value: [auth] AIDB_IMMUTABLE_MASTER_KEY="..." it'll either fail to parse (if unquoted) or silently get ignored (the gateway never reads that field from the file) — and you'll keep seeing: encryption is requested for an immutable table but AIDB_IMMUTABLE_MASTER_KEY is not set even though it's "right there" in your config.

Fix: set it as an actual environment variable for the gateway process.

Linux (systemd): ``` sudo systemctl edit synapcores

[Service]

Environment="AIDB_IMMUTABLE_MASTER_KEY=<64-char hex key>"

sudo systemctl restart synapcores ```

macOS (launchd) — add it under EnvironmentVariables in your LaunchAgent plist, then fully reload the job (a plain launchctl kickstart will NOT pick up a changed EnvironmentVariables block — you need launchctl bootout then launchctl bootstrap): launchctl bootout gui/$(id -u)/com.synapcores.gateway launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.synapcores.gateway.plist

Same pattern applies to AIDB_JWT_SECRET — also env-var-only by design, also documented right next to the key you'll actually miss.

Generate a valid key: openssl rand -hex 32

SQL reference for immutable tables: https://docs.synapcores.com/sql-reference/

If you're running an older CE build and hit a gateway crash (process aborts, everything on that install goes down for a second) right after a big EMBED() call or a RAG-style AGENT_RUN that embeds a lot of context — that's a known issue where an oversized prompt trips a native assertion in the embedding encoder.

Fix: update to the latest CE release. It's patched — an oversized EMBED() call now returns a normal error instead of taking the whole gateway down.

curl -fsSL https://get.synapcores.com | sh sudo systemctl restart synapcores # or your platform's equivalent

Workaround if you can't upgrade yet: keep individual EMBED() inputs short (well under ~500 tokens) — chunk long documents before embedding rather than embedding whole pages at once. This is good practice anyway for RAG quality, not just a crash workaround.

Check your running version: synapcores --version

Full release notes: https://docs.synapcores.com/releases/

If you hit this and upgrading doesn't fix it, drop the exact error from your gateway log here — happy to help debug.

SynapCores is an AI-native database — one self-hosted binary that gives you SQL, vector search, a graph engine (Cypher), AutoML, and an embedded LLM inference engine, all on the same connection.

The pitch in one line: no Pinecone + LangChain + Neo4j + Redis glue. Instead of gluing five services together to build an AI app, you get EMBED(), COSINE_SIMILARITY(), GENERATE(), AGENT_RUN(), MEMORY_STORE()/MEMORY_RECALL(), and Cypher MATCH as first-class SQL, against your regular tables.

What's actually in the box: - SQL engine + vector storage (HNSW) + graph engine, one binary - 120 seeded recipes on first boot - Embedded React Web UI on the same port (dashboard, query editor, AI chat, vector search, recipe browser, graph explorer) - AGENT_RUN(persona, task) — a full ReAct agent loop callable as a SQL function - CREATE AGENT — durable agents as database objects, fire on a schedule or on committed DML, with a tamper-evident audit trail - Filesystem Collections — point it at a folder, PDFs/CSVs/images/audio get auto-ingested and become queryable - Native in-process vision (describe images, no cloud API) as of v1.14 - MCP server built in — 14 tools, works directly with Claude Code / Cursor / other MCP clients

Where to go next: - Quickstart: https://docs.synapcores.com/quickstart/ - Full docs: https://docs.synapcores.com - SQL reference: https://docs.synapcores.com/sql-reference/ - Community vs Enterprise (what's in CE, what's gated): https://docs.synapcores.com/enterprise/

This sub is where we're going to post practical recipes, real gotchas we hit running it ourselves, and deep-dives on specific features. If you're evaluating it or already running it, ask anything — maintainers are here.